Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

hayato

hayato

Not a member

hayato's latest activity

Friday, October 30 2009
12 Thread(0x100190360): deadlock (fatal) was updated in Anemone
  • hayato
    hayato commented at 11:31 PM

    I have a similar experience of this ticket.

    Throwing exception from my code in result.

    Please add following flag to your code and retry.
    If error in your code, ...
Wednesday, September 16 2009
9 limit-depth crawling was updated in Anemone
  • hayato
    hayato commented at 9:52 AM

    This ticket looks fixed at anemone 0.2.0.

    thanks!
10 following <img> tag was created in Anemone
  • hayato
    hayato created the ticket at 9:51 AM

    I suggest adding feature to img tag tracking.

    • add :follow_img_tag option.
    • following img tag if :follow_img_tag => true.
    • add body attribute to Page to get raw h...
Sunday, September 06 2009
9 limit-depth crawling was created in Anemone
  • hayato
    hayato created the ticket at 12:31 AM

    I suggest to add new function to anemone.

    Problem
    Anemone 0.1.2 follow link in same domain. In some of the cases of root URL, Anemone crawl too many pages, and ...
Monday, August 24 2009
6 crawling slowly was updated in Anemone
  • hayato
    hayato commented at 4:25 AM

    Your fix looks good.

    But, following test is failed.

    .F.FF.

1)
'Anemone::Page should store the response headers when fetching a page' FAILED
expected nil? to r...
8 fail test by autotest was created in Anemone
  • hayato
    hayato created the ticket at 4:04 AM

    Unittest of anemone 0.1.2 is written by RSpec.
    but test by autotest (included ZenTest) is failed.

    [~/git/anemone]> autospec
loading autotest/rspec
/opt/local/b...
Tuesday, August 18 2009
7 customizable page/link queue was created in Anemone
  • hayato
    hayato created the ticket at 5:50 PM

    I suggest allowing customize queue.

    Current implementation page/link queue is instance of Queue object.
    But exist following use case.

    • use Message Queuing softw...
Tuesday, August 04 2009
6 crawling slowly was updated in Anemone
  • hayato
    hayato commented at 4:50 PM

    sorry my broken comment......

    Term "reserved" in patch causes confusion. "Delay" is felicity term.
    I attached the new patch.

    if apply this patch,this code behav...
  • hayato
    hayato commented at 4:41 PM

    Term "reserved" in patch causes confusion.
    "Delay" is felicity term.

    I attached the new patch.

    if apply this patch,this code behavior improve.

    require 'anemone'
    ...
Monday, August 03 2009
6 crawling slowly was updated in Anemone
  • hayato
    hayato commented at 5:11 PM

    patch attatched

    add on_pre_fetch handler.

    if this block return false,link is not fetched and re-enq to link_queue.