Remove duplicate entries from requirements.txt

[shiwani42]

Summary

Fixes #853

requirements.txt had accumulated duplicate package entries from separate commits that each added packages without checking for existing entries.

Removed duplicates:

• playwright (second occurrence)
• psycopg2-binary (second occurrence)
• pygithub (kept PyGithub)
• python_markdown_maker (kept python-markdown-maker)
• scikit_learn (kept scikit-learn)
• setuptools (second occurrence)
• SQLAlchemy (second occurrence)
• compliance-trestle (second occurrence)

neo4j and neomodel were verified as distinct packages and are not affected.

Test plan

• sort requirements.txt | uniq -di returns no output
• pip install -r requirements.txt completes without conflicts

[Pa04rth]

@shiwani42 Hey Shiwani , before raising a PR , please pin Spyros or me to validate the issue , you are trying to solve ? This way we might give you some suggestion to make the issue more reasonable and important .
Please read the contribution guide !

[Pa04rth]

@shiwani42 I want to ask you in this PR how we will get to know which version of the package is actually in use ! which one is relevant ?

[shiwani42]

@shiwani42 Hey Shiwani , before raising a PR , please pin Spyros or me to validate the issue , you are trying to solve ? This way we might give you some suggestion to make the issue more reasonable and important . Please read the contribution guide !

Thanks for the heads up, will ping before raising PRs next time. Sorry for skipping that.

[shiwani42]

@shiwani42 I want to ask you in this PR how we will get to know which version of the package is actually in use ! which one is relevant ?

All duplicates are unpinned on both sides, so no version ambiguity. pip resolves to the same version regardless. The removal just cleans up the redundant lines.

[Pa04rth]

@shiwani42 agreed that since the dependencies are unpinned, removing redundant entries won’t change how pip resolves them.

That said, I think this highlights a bigger issue in our setup: since we don’t pin versions in requirements.txt, the environment is not deterministic. Pip will install the latest compatible versions each time, which means different developers (or CI) could end up with different versions and potentially different behavior or breakages.

To make this more reliable, we should consider locking dependencies to exact versions (e.g., using pip freeze or a tool like pip-tools/poetry). This way we ensure reproducible builds and avoid unexpected issues in the future.

Would you be open to adding pinned versions as a follow-up?

[shiwani42]

Thanks for the context! I'll close this PR and open a new issue specifically for pinning the dependencies to exact versions, then raise a fresh PR for that.

[Pa04rth]

@shiwani42 I am closing the issue , reopen a new one with updated context