importing .ICS on the with_heroku branch

[max-mapper]

hiya!

I'm trying to get a calendar set up for the oakland tech community. I have it running here: http://oaklandar.herokuapp.com

However, I get ERROR: Unable to import: Unknown error: Validation failed: Title can't be blank when trying to import a .ics URL from a public google calendar

now im trying to figure out how to run source_parser_ical_spec.rb locally to see if it passes on the with_heroku but I don't have postgres installed. is there a way to run that test in isolation? I would also be grateful for any tips on getting ICS import working

[max-mapper]

p.s. here is the .ics url I was using:
https://www.google.com/calendar/ical/e0a0c178tfj07mr30k9n35sf2nnn45ie%40import.calendar.google.com/public/basic.ics

[wrought]

The calendar is here http://sudoroom.org/calendar/ with an export here webcal://sudoroom.org/?plugin=all-in-one-event-calendar&controller=ai1ec_exporter_controller&action=export_events&cb=966118952

I believe Max piped this webcal export thorugh a google calendar to get an .ICS file.

I updated a location-missing event with a non-null location, but still got:

"ERROR: Unable to import: Unknown error: Validation failed: Title can't be blank"

I see no 'Title' element of the iCal spec... thinking this is an error from Calagator.

[max-mapper]

someone in #pdxtech IRC told me they got the calagator model specs to pass
with the sudoroom .ics data used as a fixture IF they removed the sound art
meetup because the 'Title can't be blank' referred to the "location title".
I dont see a location title or a title in the .ical data though.

On Mon, Feb 4, 2013 at 9:57 AM, wrought notifications@github.com wrote:

The calendar is here http://sudoroom.org/calendar/ with an export here
webcal://
sudoroom.org/?plugin=all-in-one-event-calendar&controller=ai1ec_exporter_controller&action=export_events&cb=966118952

I believe Max piped this webcal export thorugh a google calendar to get an
.ICS file.

I updated a location-missing event with a non-null location, but still got:

"ERROR: Unable to import: Unknown error: Validation failed: Title can't be
blank"

I see no 'Title' element of the iCal spec... thinking this is an error
from Calagator.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/25#issuecomment-13089673.

[max-mapper]

ahha, calagator uses the 'summary' property in ical for the event.title
https://github.com/calagator/calagator/blob/master/app/models/source_parser/ical.rb#L92

now to figure out why it fails when there are summaries on all events in
the ical data...

On Mon, Feb 4, 2013 at 10:00 AM, Max Ogden max@maxogden.com wrote:

someone in #pdxtech IRC told me they got the calagator model specs to pass
with the sudoroom .ics data used as a fixture IF they removed the sound art
meetup because the 'Title can't be blank' referred to the "location title".
I dont see a location title or a title in the .ical data though.

On Mon, Feb 4, 2013 at 9:57 AM, wrought notifications@github.com wrote:

The calendar is here http://sudoroom.org/calendar/ with an export here
webcal://
sudoroom.org/?plugin=all-in-one-event-calendar&controller=ai1ec_exporter_controller&action=export_events&cb=966118952

I believe Max piped this webcal export thorugh a google calendar to get
an .ICS file.

I updated a location-missing event with a non-null location, but still
got:

"ERROR: Unable to import: Unknown error: Validation failed: Title can't
be blank"

I see no 'Title' element of the iCal spec... thinking this is an error
from Calagator.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/25#issuecomment-13089673.

[revmischa]

You do not need to run Pg locally to test things, the Gemfile should have:

group :production, :staging do
  gem "pg"
end

group :development, :test do
  gem "sqlite3", "~> 1.3.0", :require => "sqlite3"
end

[matthewg42]

Was a solution to this ever found? I am trying to set up a calagator instance, and the first thing I did was point it at a google-based ical feed, and got the same error. The feed I am using is:

https://www.google.com/calendar/ical/info%40nottinghack.org.uk/public/basic.ics

And I'm getting the same error message.

ERROR: Unable to import: Unknown error: Validation failed: Title can't be blank

Just cloned the code today.

[max-mapper]

I haven't found a solution yet. If you know a rails dev with some free time
for open source you should point em here!

On Thu, Apr 18, 2013 at 1:44 PM, Matthew Gates notifications@github.comwrote:

Was a solution to this ever found? I am trying to set up a calagator
instance, and the first thing I did was point it at a google-based ical
feed, and got the same error. The feed I am using is:

https://www.google.com/calendar/ical/info%40nottinghack.org.uk/public/basic.ics

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/25#issuecomment-16607545
.

[matthewg42]

I'm trying to learn rails, and I plan to have a look at it, but I don't know if my knowledge is good enough yet to figure it out. I'll post anything I find to this thread.

[matthewg42]

Not solved yet, but I have some more information. The problem occurs when event.save! is called in source.create_events. An ActiveRecord::RecordInvalid exception is thrown. I extracted the following backtrace (line numbers will be a little off in the app code because I added bunch of logging). I'm not entirely sure how to interpret this:

/opt/local/lib/ruby1.9/gems/1.9.1/gems/activerecord-3.2.12/lib/active_record/attribute_methods/dirty.rb:33:in `save!'
/opt/local/lib/ruby1.9/gems/1.9.1/gems/activerecord-3.2.12/lib/active_record/transactions.rb:264:in `block in save!'
/opt/local/lib/ruby1.9/gems/1.9.1/gems/activerecord-3.2.12/lib/active_record/transactions.rb:313:in `block in with_transaction_returning_status'
/opt/local/lib/ruby1.9/gems/1.9.1/gems/activerecord-3.2.12/lib/active_record/connection_adapters/abstract/database_statements.rb:192:in `transaction'
/opt/local/lib/ruby1.9/gems/1.9.1/gems/activerecord-3.2.12/lib/active_record/transactions.rb:208:in `transaction'
/opt/local/lib/ruby1.9/gems/1.9.1/gems/activerecord-3.2.12/lib/active_record/transactions.rb:311:in `with_transaction_returning_status'
/opt/local/lib/ruby1.9/gems/1.9.1/gems/activerecord-3.2.12/lib/active_record/transactions.rb:264:in `save!'
/Users/mouse/Projects/rails/calagator/app/models/source.rb:92:in `block in create_events!'
/Users/mouse/Projects/rails/calagator/app/models/source.rb:71:in `each'
/Users/mouse/Projects/rails/calagator/app/models/source.rb:71:in `create_events!'
/Users/mouse/Projects/rails/calagator/app/controllers/sources_controller.rb:21:in `import'
/opt/local/lib/ruby1.9/gems/1.9.1/gems/actionpack-3.2.12/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/opt/local/lib/ruby1.9/gems/1.9.1/gems/actionpack-3.2.12/lib/abstract_controller/base.rb:167:in `process_action'
/opt/local/lib/ruby1.9/gems/1.9.1/gems/actionpack-3.2.12/lib/action_controller/metal/rendering.rb:10:in 
...

[matthewg42]

It seems to be something in the ICS parser itself - specifically handling HTML tags and escapes in descriptions can trigger it.

For example, look at these two feeds:

• http://porpoisehead.net/cal/ok.ics
• http://porpoisehead.net/cal/bad.ics

The only difference is that bad has a description containing a bunch of HTML and other stuff. What's weird is that this event isn't even active (the date is in the past), but it still prevents the good event being added.

Yay, progress!

[matthewg42]

I think the problem is multi-byte characters in the feed. When I ran it through a UTF -> ASCII converter, the whole giant feed loaded just fine. The £ symbol in the description of the second item in the tests above seems to be what did it.

You can see a full, working feed here: http://porpoisehead.net/cal/ascii.ics

[wrought]

Wow, nice work--think a patch to handle Unicode will be possible, or is converting to ASCII necessary?

Cheers!

// Matt

----- Reply message -----
From: "Matthew Gates" notifications@github.com
To: "calagator/calagator" calagator@noreply.github.com
Cc: "wrought" mattsenate@gmail.com
Subject: [calagator] importing .ICS on the with_heroku branch (#25)
Date: Wed, Apr 24, 2013 9:00 AM
I think the problem is multi-byte characters in the feed. When I ran it through a UTF -> ASCII converter, the whole giant feed loaded just fine. The £ symbol in the description of the second item in the tests above seems to be what did it.

You can see a full, working feed here: http://porpoisehead.net/cal/ascii.ics

—
Reply to this email directly or view it on GitHub.

[matthewg42]

My feeling is that UTF-8 should be legal in text fields, or at least Calagator shouldn't refuse to import everything else if it finds some. I'm trying to work out where this is done at the moment. I assume in app/models/source_parser somewhere. I'll look at it again shortly and create a patch if I can, or just comment here if I can't work it out.

[max-mapper]

I haven't done ruby in a few years but here is some code for converting UTF8 to ASCII that may be useful here: https://github.com/maxogden/shp2geocouch/blob/master/bin/shp2geocouch#L92-L93

I would take a crack at implementing it here but I'd need someone to give me a command or set of commands I can run that will run the source_parser model tests (from scratch, assuming I have 0 rails experience -- which is pretty close to true nowadays)

[shawnacscott]

Is this still an existing issue?