CluedIn External Search for Website Crawler.
This repository contains the code and associated tests for crawling a public website based of Entities and Clues that have set a value for the Organization.Website core vocabulary.
To use the Web External Search with the CluedIn server you will have to add the CluedIn.Enricher.Web nuget package to your environment.
A mocked environment is required to run integration and acceptance tests. The mocked environment can be built and run using the following Docker command:
docker-compose up --build -dUse the following commands to run all Unit and Integration tests within the repository:
dotnet test .\ExternalSearch.Web.sln --filter Unit
dotnet test .\ExternalSearch.Web.sln --filter IntegrationTo run Pester acceptance tests
invoke-pesterTo review the WireMock HTTP proxy logs
docker-compose logs wiremockCluedIn is the Cloud-native Master Data Management Platform that brings data teams together enabling them to deliver the foundation of high-quality, trusted data that empowers everyone to make a difference.
We're different because we use enhanced data management techniques like Graph and Zero Upfront Modelling to accelerate the time taken to prepare data to deliver insight by as much as 80%. Installed in as little as 20 minutes from the Azure Marketplace, CluedIn is fully integrated with Microsoft Purview and the full Microsoft Fabric suite, making it the preferred choice for Azure customers.
To learn more about CluedIn, contact the team today.