Skip to content

Crawlabout v2.2.0

Latest

Choose a tag to compare

@github-actions github-actions released this 05 Nov 19:31

Added

  • Exports show their progress: a bar at the bottom right of the window,
    with a button to cancel. For a large check a PDF or HTML report with
    every URL takes a while.

Changed

  • Every report (JSON, CSV, HTML, PDF and the list exports) is sent while
    it is written: the download starts at once instead of after the whole
    file was created, and Crawlabout no longer needs memory for the whole
    file.
  • Large websites (100,000 URLs and more) no longer bring the browser to a
    halt. The page used to load every result several times a second while a
    check ran; now the server filters, sorts and counts, and the page only
    loads what it shows. During a check it gets updates pushed instead of
    asking for them.
  • The results list is shown in pages of 250 entries, with buttons to
    page through it. Filters, sorting and the content search work on all
    results, and "Export list" still exports every page of the list.
  • Graph tab: with more than 2,500 URLs the graph groups them by directory
    on its own, because the browser cannot draw that many pages one by one
    smoothly. The checkbox "Group by directory" is then on and greyed out;
    its tooltip says why. If all pages are in one directory, the graph shows
    the 2,500 pages closest to the start page, and the legend says how many
    are left out.
  • The content search uses all processor cores, and while a check runs it
    only searches the pages added since. The link graph of a large check is
    built about ten times faster, and Crawlabout needs less memory for the
    links between the pages.
  • Graph tab: a click on a page shows the pages it links to (one or two
    links away) instead of every page linking to it or from it, plus the
    shortest click path from the start page as before.
  • Graph tab: names appear only for the dot under the mouse pointer. Its
    neighbours are highlighted without names; the names of all larger dots,
    groups and neighbours used to cover each other.

Fixed

  • The password fields under "Advanced settings" (check and offline copy)
    look like the other input fields again.