Configuration

All configurations can be modified through config.py. You can follow instructions below to configure each default module. Please note this is only for users to set up the system, which does NOT include any instructions for module customization. To build your own module, you should follow advanced guides for developers.

LLM

By default, the LLM module calls OpenAI Chat service using GPT-3.5 model with a temperature value of 0. Some configs accepts value by export environment variables like export ARG_NAME=arg_value.

Argument	Required	Data Type	Supported Value	Env Variable
LLM_OPTION	yes	str	openai (default), ernie, minimax, dolly, skychat, dashscope, chatglm	LLM_OPTION
CHAT_CONFIGS	yes	dict	key: a llm option value: a dict of llm configs	See options below for each LLM_OPTION

You can modify the following configs via setting environment variables:

# openai
$ export OPENAI_API_KEY=your_openai_api_key

# ernie
$ export ERNIE_API_KEY=your_ernie_api_key
$ export ERNIE_SECRET_KEY=your_ernie_secret_key

# minimax
$ export MINIMAX_API_KEY=your_minimax_api_key
$ export MINIMAX_GROUP_ID=your_minimax_group_id

# skychat
$ export SKYCHAT_API_HOST=your_skychat_api_host  # default value: 'sky-api.singularity-ai.com'
$ export SKYCHAT_APP_KEY=your_skychat_app_key
$ export SKYCHAT_APP_SECRET=your_skychat_app_secret

# dashscope
$ export DASHSCOPE_API_KEY=your_dashscope_api_key

# chatglm
$ export ZHIPUAI_API_KEY=your_zhipuai_api_key

Click to expand more details of CHAT_CONFIG.

CHAT_CONFIG = {
    'openai': {
        'openai_model': 'gpt-3.5-turbo',
        'openai_api_key': None,  # will use environment  value 'OPENAI_API_KEY' if None
        'llm_kwargs': {
            # 'temperature': 0.8,
            # 'max_tokens': 200,
            }
    },
    'ernie': {
        'ernie_api_key': None, # If None, use environment  value 'ERNIE_API_KEY'
        'ernie_secret_key': None, # If None, use environment value 'ERNIE_SECRET_KEY'
        'llm_kwargs': {}
    },
    'minimax': {
        'minimax_model': 'abab5-chat',
        'minimax_api_key': None, # If None, use environment value 'MINIMAX_API_KEY'
        'minimax_group_id': None, # If None, use environment value 'MINIMAX_GROUP_ID'
        'llm_kwargs': {}
    },
    'dolly': {
        'dolly_model': 'databricks/dolly-v2-3b',
        'llm_kwargs': {'device': 'auto'}
    },
    'skychat': {
        'skychat_api_host': None, # If None, use default value 'sky-api.singularity-ai.com'
        'skychat_app_key': None, # If None, use environment value 'SKYCHAT_APP_KEY'
        'skychat_app_secret': None  # If None, use environment value 'SKYCHAT_APP_SECRET'
    },
    'dashscope': {
        'dashscope_model': 'qwen-plus-v1',
        'dashscope_api_key': None  # If None, use environment value 'DASHSCOPE_API_KEY'
    },
    'chatglm':{
        'chatglm_model': 'chatglm_130b',
        'chatglm_api_key': None  # If None, use environment value 'ZHIPUAI_API_KEY'
    }
}

Embedding

By default, the embedding module uses LangChain SentenceTransformersEmbeddings to convert text inputs to vectors.

• model: multi-qa-mpnet-base-cos-v1(420MB)
• dim: 768
• normalization: True

If you want to change model or disable normalization, you can modify the embedding section in configuration file.

TEXTENCODER_CONFIG = {
    'model': 'multi-qa-mpnet-base-cos-v1',
    'norm': True, # Set as False to disable extra normalization after embedding
    'dim': 768
}

You can find more model names for sentence embedding at SBert.

Store

Configurations for each store service is set up via config.py under store directory. You need to check if your services are available using default values:

• Vector database:

  VECTORDB_CONFIG = {
      'connection_args': {
          'uri': os.getenv('ZILLIZ_URI', 'https://localhost:19530'),
          'token': os.getenv('ZILLIZ_TOKEN', None)
          },
      'top_k': 10,
      'threshold': 0.6,
      'index_params': {
          'metric_type': 'IP',
          'index_type': 'IVF_FLAT',
          'params': {'nlist': 1024}
          }
  }

• Scalar database:

  # Set True to enable use of scalar db
  USE_SCALAR = True if os.getenv('USE_SCALAR', False).lower() == 'true' else False
  
  SCALARDB_CONFIG = {
      'connection_args': {
          'cloud_id': os.getenv('ES_CLOUD_ID'),
          'ca_certs': os.getenv('ES_CA_CERTS', None),
          'basic_auth': (os.getenv('ES_USER', 'user_name'), os.getenv('ES_PASSWORD', 'es_password'))
          },
      'top_k': 5
  }

• SQL database for Memory:

  MEMORYDB_CONFIG = {
      'connect_str': os.getenv('SQL_URI', 'sqlite:///./sqlite.db')
  }

You can also configure the following arguments via environment argument:

# Set up Vector Store (eg. Milvus)
$ export ZILLIZ_URI=http://localhost:19530 # replace with your uri
$ export ZILLIZ_TOKEN=your_zilliz_api_key  # skip if none

# Set up Scalar Store (eg. Elastic)
$ export USE_SCALAR=True
$ export ES_HOSTS=https://localhost:9200
$ export ES_CA_CERTS=path/to/es_ca_certs
$ export ES_USER=your_es_user
$ export ES_PASSWORD=your_es_password

# Set up Memory Store (eg. SQLite)
$ export SQL_URI=sqlite:///./sqlite.db

DataLoader

By default, DataParser splits a document into chunks with size of 300 and uses GPT-3.5 to extract questions.

To configure the module, you can modify the DataLoader section in configuration.

DATAPARSER_CONFIG = {
    'chunk_size': 300
}

QUESTIONGENERATOR_CONFIG = {
    'model_name': 'gpt-3.5-turbo',
    'temperature': 0,
}