This is a simple file indexing API server which gives a utility to search through files in a selected directory.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
cmd/dexer
lib
scripts
ui
vendor
.gitignore dexer: Change the project name & org to dexer and dgplug Jan 12, 2019
.travis.yml
Dockerfile
Gopkg.lock Switched to Go Modules Nov 22, 2018
Gopkg.toml
LICENSE
Makefile
README.md Fix read me for docker Jan 14, 2019
changelog.md
config.json Updated the config file according to the new configuration Nov 11, 2018
go.mod
go.sum Added go.sum for reproducible builds Dec 27, 2018

README.md

File Indexer Build Status

This is a simple project which reads files in the directory mentioned in the config file and indexes it. This then can be used to search the files present in the directory.

Motivation

This is a way of trying to index the logs file in dgplug so that we can easily find the files in which a word has occured in the file.

Dependency

This assumes that you have Go installed and setup.

pip3 and python3 is what we need for development purpose.

How to install

Note : Right now we are unable to provide binary builds, so have to build the program the regular way or you can build a docker container.

  1. Install git.
  2. Install Go (need a version which supports Go Modules).
  3. Run the following commands :
$ git clone https://github.com/dgplug/dexer.git
$ cd dexer
$ make all
$ dexer

To use the program we need a configuration file which is provided by the config.json file. Here is an example of one:

{
    "RootDirectory": "logs",
    "IndexFilename": "irclogs.bleve",
    "Port": ":8000",
    "LogFile": "logfile"
}

There are 4 entries :

  • RootDirectory : Location of the logs which we want to search through.
  • IndexFilename : The file where bleve will store all the information related to indexing.
  • Port : The port on which the server will be run.
  • LogFile : The file which will be used to store the logs.

API

Once everything is working fine install the postman plugin for your browser. And from that plugin you need to hit the endpoint as:

localhost:<port>/search/american

Here, port number passed to the program using the configuration file ,american is the query word that I passed, make sure to open any file in the logs/ directory and find a word to search. It will look like this:

Missing screenshot

Use the Web Front End

You can also visit localhost:<port> to use the web frontend which comes with the program to search.

Web Front End

Run locally using docker

You can run the application using Docker in your local machine. It will use the Dockerfile instructions. Make sure you have Docker installed in your machine.

Run the following commands to build and run the docker image.

$ git clone https://github.com/dgplug/dexer.git
$ cd dexer
$ make logs
$ make docker-build # builds docker image
$ make docker-run # runs the image in new container

One has to make sure the logs directory has all the file because dexer runs the indexing at the starting and then keeps it, so if a file is not there it would not index it.