Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Workplace Search connector #991

Merged
merged 82 commits into from Dec 22, 2020
Merged

Add Workplace Search connector #991

merged 82 commits into from Dec 22, 2020

Commits on Jul 20, 2020

  1. Configuration menu
    Copy the full SHA
    a75fa0c View commit details
    Browse the repository at this point in the history
  2. Rename to workplace search

    dadoonet committed Jul 20, 2020
    Configuration menu
    Copy the full SHA
    7780e96 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    1293f38 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    97a422d View commit details
    Browse the repository at this point in the history
  5. Add the first "manual" IT for WPSearch

    It's just a skeleton for now.
    dadoonet committed Jul 20, 2020
    Configuration menu
    Copy the full SHA
    04181b2 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2020

  1. Initial step

    This one is working manually.
    dadoonet committed Jul 29, 2020
    Configuration menu
    Copy the full SHA
    09c4265 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2020

  1. Add tests for all documents

    dadoonet committed Jul 30, 2020
    Configuration menu
    Copy the full SHA
    7a0692e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    73248dc View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2020

  1. Add a Service abstraction layer

    Make more tests to pass
    dadoonet committed Jul 31, 2020
    Configuration menu
    Copy the full SHA
    d893310 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    4d5e8e9 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2020

  1. Add the delete method

    dadoonet committed Aug 3, 2020
    Configuration menu
    Copy the full SHA
    136b71a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    bca1ff7 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3d28682 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    da74dd1 View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2020

  1. Add an internal WPSearch Client

    Until we have an official one
    dadoonet committed Aug 4, 2020
    Configuration menu
    Copy the full SHA
    a91f65a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    32ad3d6 View commit details
    Browse the repository at this point in the history
  3. Add a comment (TODO)

    dadoonet committed Aug 4, 2020
    Configuration menu
    Copy the full SHA
    2685c5a View commit details
    Browse the repository at this point in the history
  4. Fix unit test

    dadoonet committed Aug 4, 2020
    Configuration menu
    Copy the full SHA
    3f9d972 View commit details
    Browse the repository at this point in the history
  5. Add a skeleton for search

    dadoonet committed Aug 4, 2020
    Configuration menu
    Copy the full SHA
    bfbb0df View commit details
    Browse the repository at this point in the history
  6. Fix failing test

    dadoonet committed Aug 4, 2020
    Configuration menu
    Copy the full SHA
    f3c1fbb View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    0262c39 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    23652ab View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    8b6c5aa View commit details
    Browse the repository at this point in the history
  10. Fix license headers

    dadoonet committed Aug 4, 2020
    Configuration menu
    Copy the full SHA
    35c7006 View commit details
    Browse the repository at this point in the history
  11. Fix failing test

    dadoonet committed Aug 4, 2020
    Configuration menu
    Copy the full SHA
    b7c1387 View commit details
    Browse the repository at this point in the history
  12. Fix failing tests

    dadoonet committed Aug 4, 2020
    Configuration menu
    Copy the full SHA
    0d873b3 View commit details
    Browse the repository at this point in the history
  13. Fix failing tests

    dadoonet committed Aug 4, 2020
    Configuration menu
    Copy the full SHA
    1244127 View commit details
    Browse the repository at this point in the history
  14. Fix NPE

    dadoonet committed Aug 4, 2020
    Configuration menu
    Copy the full SHA
    fc863da View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    c02250d View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2020

  1. Fix typo

    dadoonet committed Aug 5, 2020
    Configuration menu
    Copy the full SHA
    a97cc10 View commit details
    Browse the repository at this point in the history
  2. Add support for running tests against Cloud

    And associated documentation
    dadoonet committed Aug 5, 2020
    Configuration menu
    Copy the full SHA
    8c56b56 View commit details
    Browse the repository at this point in the history
  3. Fix failing test

    dadoonet committed Aug 5, 2020
    Configuration menu
    Copy the full SHA
    d2b30df View commit details
    Browse the repository at this point in the history
  4. Fix failing test

    dadoonet committed Aug 5, 2020
    Configuration menu
    Copy the full SHA
    1d2257f View commit details
    Browse the repository at this point in the history
  5. Fix failing test

    dadoonet committed Aug 5, 2020
    Configuration menu
    Copy the full SHA
    4daec54 View commit details
    Browse the repository at this point in the history
  6. Fix imports

    dadoonet committed Aug 5, 2020
    Configuration menu
    Copy the full SHA
    fb5a4ee View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2020

  1. Add a BulkProcessor mechanism for WPSearch

    Also fix failing tests when running on cloud
    Also fix an issue with cloud ids
    Replace the `file://` url by a temporary `http://fscrawler-repository` URL. This will be configurable in the future.
    dadoonet committed Aug 6, 2020
    Configuration menu
    Copy the full SHA
    728eec2 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a086418 View commit details
    Browse the repository at this point in the history
  3. Fix failing test

    dadoonet committed Aug 6, 2020
    Configuration menu
    Copy the full SHA
    df49835 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    72da79e View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    dcbb34f View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2020

  1. Configuration menu
    Copy the full SHA
    065d381 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    52acce6 View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2020

  1. Configuration menu
    Copy the full SHA
    d69c076 View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2020

  1. WIP: Update demo to 7.9.1

    dadoonet committed Sep 7, 2020
    Configuration menu
    Copy the full SHA
    4833c16 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    054b303 View commit details
    Browse the repository at this point in the history
  3. FSCrawler is not closing properly

    We called twice the run of the Thread. Which causes duplicates and avoid closing the crawler.
    dadoonet committed Sep 7, 2020
    Configuration menu
    Copy the full SHA
    6556b31 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2020

  1. Use filename as title when no title is available

    For Workplace Search, we use the name of the file if no title has been found in the document metadata.
    dadoonet committed Sep 14, 2020
    Configuration menu
    Copy the full SHA
    167666d View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    356fa5f View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    c018d16 View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2020

  1. Configuration menu
    Copy the full SHA
    742f06b View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2020

  1. Switch to Docker-Compose instead of Docker for tests

    We are switching from `docker-maven-plugin` to `docker-compose-maven-plugin` to have a better way to test our full platform.
    That way we can also provide to the community a docker-compose contrib example which we are using within our own tests.
    
    Note that this is using a not released yet version for the maven plugin waitfor: https://github.com/simplaex/waitfor-maven-plugin/
    dadoonet committed Sep 24, 2020
    Configuration menu
    Copy the full SHA
    b67d2ad View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2020

  1. Configuration menu
    Copy the full SHA
    10b16b1 View commit details
    Browse the repository at this point in the history
  2. Update to waitfor plugin 1.3

    dadoonet committed Sep 28, 2020
    Configuration menu
    Copy the full SHA
    f89103c View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2020

  1. Configuration menu
    Copy the full SHA
    87488ec View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2020

  1. Configuration menu
    Copy the full SHA
    7f0c605 View commit details
    Browse the repository at this point in the history
  2. Add WPSearchAdminClient

    For that we are simulating a end user clicking on the web interface to create custom sources...
    It's a temporary solution waiting for a clean API from Workplace Search project.
    dadoonet committed Oct 12, 2020
    Configuration menu
    Copy the full SHA
    470191f View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2020

  1. Configuration menu
    Copy the full SHA
    252c925 View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2020

  1. Merge branch 'master' into wip/workplace_search

    # Conflicts:
    #	.gitignore
    #	cli/src/main/resources/log4j2.xml
    #	core/src/main/java/fr/pilato/elasticsearch/crawler/fs/FsParserAbstract.java
    #	docs/source/fscrawler.ini
    #	pom.xml
    dadoonet committed Nov 13, 2020
    Configuration menu
    Copy the full SHA
    6157be9 View commit details
    Browse the repository at this point in the history
  2. Bump elasticsearch-rest-high-level-client from 7.9.3 to 7.10.0

    Bumps [elasticsearch-rest-high-level-client](https://github.com/elastic/elasticsearch) from 7.9.3 to 7.10.0.
    - [Release notes](https://github.com/elastic/elasticsearch/releases)
    - [Commits](elastic/elasticsearch@v7.9.3...v7.10.0)
    
    Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
    dadoonet committed Nov 13, 2020
    Configuration menu
    Copy the full SHA
    d428d22 View commit details
    Browse the repository at this point in the history
  3. Bump elasticsearch-rest-high-level-client from 7.9.3 to 7.10.0

    Bumps [elasticsearch-rest-high-level-client](https://github.com/elastic/elasticsearch) from 7.9.3 to 7.10.0.
    - [Release notes](https://github.com/elastic/elasticsearch/releases)
    - [Commits](elastic/elasticsearch@v7.9.3...v7.10.0)
    
    Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
    dadoonet committed Nov 13, 2020
    Configuration menu
    Copy the full SHA
    4d54d59 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    8385b0e View commit details
    Browse the repository at this point in the history

Commits on Nov 16, 2020

  1. Fix NPE in tests

    dadoonet committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    6eff7e5 View commit details
    Browse the repository at this point in the history

Commits on Nov 17, 2020

  1. Use ent_search.auth.native1.source=elasticsearch-native

    With that, we will be ready for 7.11 which supports from Docker Compose settings having Workplace Search available from Kibana.
    dadoonet committed Nov 17, 2020
    Configuration menu
    Copy the full SHA
    344098d View commit details
    Browse the repository at this point in the history

Commits on Nov 19, 2020

  1. Documentation issue

    dadoonet committed Nov 19, 2020
    Configuration menu
    Copy the full SHA
    ea49134 View commit details
    Browse the repository at this point in the history

Commits on Nov 23, 2020

  1. Configuration menu
    Copy the full SHA
    e8749c9 View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2020

  1. Merge branch 'master' into wip/workplace_search

    # Conflicts:
    #	tika/src/main/java/fr/pilato/elasticsearch/crawler/fs/tika/XmlDocParser.java
    dadoonet committed Nov 30, 2020
    Configuration menu
    Copy the full SHA
    c08b63a View commit details
    Browse the repository at this point in the history

Commits on Dec 2, 2020

  1. Merge branch 'master' into wip/workplace_search

    # Conflicts:
    #	beans/src/main/java/fr/pilato/elasticsearch/crawler/fs/beans/Doc.java
    #	elasticsearch-client/elasticsearch-client-base/src/test/resources/fr/pilato/elasticsearch/crawler/fs/client/fscrawler-client.properties
    #	elasticsearch-client/elasticsearch-client-base/src/test/resources/fr/pilato/elasticsearch/crawler/fs/client/v0/fscrawler-client-wrong-version.properties
    #	test-framework/src/main/java/fr/pilato/elasticsearch/crawler/fs/test/framework/AbstractFSCrawlerTestCase.java
    #	tika/src/main/java/fr/pilato/elasticsearch/crawler/fs/tika/XmlDocParser.java
    dadoonet committed Dec 2, 2020
    Configuration menu
    Copy the full SHA
    5e5c8b2 View commit details
    Browse the repository at this point in the history
  2. Fix httpmime dep

    It differs between htmlunit and tika parsers
    dadoonet committed Dec 2, 2020
    Configuration menu
    Copy the full SHA
    a4ab8d0 View commit details
    Browse the repository at this point in the history

Commits on Dec 4, 2020

  1. Configuration menu
    Copy the full SHA
    e0e52ab View commit details
    Browse the repository at this point in the history
  2. Merge branch 'master' into wip/workplace_search

    # Conflicts:
    #	integration-tests/it-common/src/main/java/fr/pilato/elasticsearch/crawler/fs/test/integration/elasticsearch/FsCrawlerRestIT.java
    #	integration-tests/it-common/src/main/java/fr/pilato/elasticsearch/crawler/fs/test/integration/elasticsearch/FsCrawlerTestRestOnlyIT.java
    dadoonet committed Dec 4, 2020
    Configuration menu
    Copy the full SHA
    51e87e1 View commit details
    Browse the repository at this point in the history
  3. Bump jersey.version from 2.32 to 3.0.0

    Bumps `jersey.version` from 2.32 to 3.0.0.
    
    Updates `jersey-container-grizzly2-http` from 2.32 to 3.0.0
    
    Updates `jersey-media-json-jackson` from 2.32 to 3.0.0
    
    Updates `jersey-media-multipart` from 2.32 to 3.0.0
    
    Updates `jersey-hk2` from 2.32 to 3.0.0
    
    Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
    dadoonet committed Dec 4, 2020
    Configuration menu
    Copy the full SHA
    6c106d8 View commit details
    Browse the repository at this point in the history

Commits on Dec 17, 2020

  1. Merge branch 'master' into wip/workplace_search

    # Conflicts:
    #	cli/src/main/java/fr/pilato/elasticsearch/crawler/fs/cli/FsCrawlerCli.java
    #	docs/source/dev/build.rst
    dadoonet committed Dec 17, 2020
    Configuration menu
    Copy the full SHA
    8fc81e5 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0ac4a96 View commit details
    Browse the repository at this point in the history

Commits on Dec 22, 2020

  1. Configuration menu
    Copy the full SHA
    dda59cb View commit details
    Browse the repository at this point in the history
  2. Disable automatic integration tests with Workplace Search

    We don't have an Admin API and all the tests to fake this API are pretty much unstable.
    
    As I want to release a version soonish, we will be running the IT with Workplace Search only manually using the CLI options `-Dtests.workplace.access_token=XXX -Dtests.workplace.key=YYY`.
    
    We might test all that using a cloud always running instance...
    dadoonet committed Dec 22, 2020
    Configuration menu
    Copy the full SHA
    d4f4bea View commit details
    Browse the repository at this point in the history
  3. Update JpegVersion

    dadoonet committed Dec 22, 2020
    Configuration menu
    Copy the full SHA
    22f0125 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    0451cf6 View commit details
    Browse the repository at this point in the history
  5. Update to 7.10.1

    dadoonet committed Dec 22, 2020
    Configuration menu
    Copy the full SHA
    86e7a52 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    b350d2e View commit details
    Browse the repository at this point in the history
  7. Fix some Sonar issues

    dadoonet committed Dec 22, 2020
    Configuration menu
    Copy the full SHA
    c4a2b79 View commit details
    Browse the repository at this point in the history
  8. Fix some Sonar issues

    dadoonet committed Dec 22, 2020
    Configuration menu
    Copy the full SHA
    d9c4450 View commit details
    Browse the repository at this point in the history
  9. Use a generic URL

    dadoonet committed Dec 22, 2020
    Configuration menu
    Copy the full SHA
    ec4f579 View commit details
    Browse the repository at this point in the history