Won't crawl without a trailing forward slash on the url
Reported by Paul Barry | July 4th, 2009 @ 09:42 AM
Something that tripped me up when I first tried to use anemone is to do:
bin/anemone_url_list.rb http://paulbarry.com
And I got back no results. I tracked it down to not having a path for the url, so entering:
bin/anemone_url_list.rb http://paulbarry.com/
Works. Here's a patch that will just set the path to '/' if there is no path.
Comments and changes to this ticket
-
Paul Barry July 4th, 2009 @ 09:46 AM
Here's the patch in a gist, lighthouse's S3 integration seems to be flaky.
-
chris (at chriskite) July 4th, 2009 @ 01:16 PM
- State changed from “new” to “resolved”
Thanks for the patch. Applied and released in 0.0.6.
Please Sign in or create a free account to add a new ticket.
With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.
Create new ticket
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป
Anemone is a Ruby library that makes it quick and painless to write programs that spider a website. It provides a simple DSL for performing actions on every page of a site, skipping certain URLs, and calculating the shortest path to a given page on a site.
chris (at chriskite)
Paul Barry