Skip to main content

Using the URL crawler to add sources.

Feedback

Hi,

When the crawler returns > 500 URLs, you have to manually remove the existing ones, which in my case was > 50. That's a very tedious and time-consuming task.

And something that will require looking for the existing ones one by one and removing them from URLs that have just been crawled.

Do you have any solutions to this? Except for removing all the sources I added earlier and completing a fresh crawl,

I truly appreciate any assistance you may offer.

Posted by Anders Eilertsen ·
Status: Completed4 comments

Log in to comment and vote

Comments4

  • Ali Mahdi

    Team•

    Sep 11, 2023

    Thanks for the feedback. We will add an option to remove all the duplicate URLs at once.

    • Anders Eilertsen

      •

      Sep 11, 2023

      Hi Ali,

      Thank you for your prompt response. I believe it would benefit the customer and user experience if duplicates could be easily removed.

      • Ali Mahdi

        Team•

        Sep 17, 2023

        Hi Anders, the feature has been implemented.

        • Anders Eilertsen

          •

          Sep 18, 2023

          I am genuinely impressed with how fast this was implemented. Thanks.