TokenHandler uses static TokenSeparatorHandler which ignores custom ITokenSeparatorProvider

[jjamid]

The TokenHandler class calls the static TokenSeparatorHandler.Handle method inside its Handle method

        private void Handle()
        {
            var c = _context.FormulaChars[_tokenIndex];
            Token tokenSeparator;
            if (CharIsTokenSeparator(c, out tokenSeparator))
            {
                if (TokenSeparatorHandler.Handle(c, tokenSeparator, _context, this))
                {
                    return;
                }

The Handle method uses a static collection which contains a MultipleCharSeparatorHandler instance

        private static SeparatorHandler[] _handlers = new SeparatorHandler[]
        { 
            new StringHandler(),
            new BracketHandler(),
            new SheetnameHandler(),
            new MultipleCharSeparatorHandler()
        };

This causes the constructor of MultipleCharSeparatorHandler to use the default TokenSeparatorProvider anyway

        public MultipleCharSeparatorHandler()
            : this(new TokenSeparatorProvider())
        {

        }

This means that it does not use the ITokenSeparatorProvider that was provided initially to the TokenHandler. e.g.

   ITokenSeparatorProvider myTokenSeparatorProvider = new MyTokenSeparatorProvider();

   var tokenFactory = new TokenFactory(myTokenSeparatorProvider, NameValueProvider.Empty, 

   FunctionNameProvider.Empty, false);
   var tokenizer = new SourceCodeTokenizer(tokenFactory, myTokenSeparatorProvider);

Suggested solution:

TokenSeparatorHandler should be non-static and accept a ITokenSeparatorProvider in its constructor to pass on to MultipleCharSeparatorHandler.

TokenHandler should initialize it inside its constructor also passing the ITokenSeparatorProvider
e.g.

         public TokenSeparatorHandler(ITokenSeparatorProvider tokenSeparatorProvider)
	 {
		    _handlers = new SeparatorHandler[]
			{
				new StringHandler(),
				new BracketHandler(),
				new SheetnameHandler(),
				new MultipleCharSeparatorHandler(tokenSeparatorProvider)
			};
	 }

         public TokenHandler(TokenizerContext context, ITokenFactory tokenFactory, 
         ITokenSeparatorProvider tokenProvider)
        {
            _context = context;
            _tokenFactory = tokenFactory;
            _tokenProvider = tokenProvider;
            _tokenSeparatorHandler = new TokenSeparatorHandler(_tokenProvider);
        

[swmal]

Agree, this looks like a design flaw. Have copied this issue to our new repo https://github.com/EPPlusSoftware/EPPlus