Add lexer

[tjysdsg]

• Implement Reader
• Add test for Reader
• Implement Lexer
• Add test for Lexer

[tjysdsg]

recognize identifiers and keywords

tan/src/lexer/lexer.cpp

Lines 156 to 161 in a932dca
// TODO: recognize identifiers and keywords
}
++current;
}
line = _reader->next_line();
}

This comment was generated by todo based on a TODO comment in a932dca in #2. cc @tjysdsg.

Done

[todo]

error handling

tan/src/lexer/lexer.cpp

Lines 216 to 221 in 5dd3518

	// TODO error handling
	}
	_token_infos.emplace_back(t);
	} else if (line[current] == '"') { // string literals
	auto *t =
	advance_for_string_literal(line, current, line_len);

---

This comment was generated by todo based on a TODO comment in 5dd3518 in #2. cc @tjysdsg.

[todo]

lex keywords

tan/src/lexer/lexer.cpp

Lines 254 to 259 in 63b534e

	// TODO: lex keywords
	_token_infos.emplace_back(t);
	} else if (line[current] == '\'') { // char
	++current;
	auto *t = new token_info;
	t->type = CHAR;

---

This comment was generated by todo based on a TODO comment in 63b534e in #2. cc @tjysdsg.

[todo]

add more test cases

tan/src/test/lexer_test.h

Lines 148 to 151 in 63b534e

	// TODO: add more test cases
	}
	#endif //__TAN_TEST_LEXER_TEST_H__

---

This comment was generated by todo based on a TODO comment in 63b534e in #2. cc @tjysdsg.