A brave attempt with several fatal issues

[Rishabh942]

First, I would like to say I think the devs trying to implement this is commendable, and my tone in the following text will parallel the most true review I can give in good conscience. It is very long since I tried to hit on every problematic aspect of my interaction with the project, but TLDR; devs are motivated but lie and falsify data, info, no evidence for claims, users have difficulty building or observing results, etc.

From testing with 3 ESPs what I've found and what a lot of other people have reported is that the technical depth seems quite impressive but the actual results with the code running on real metal vs the claims scattered throughout the README, UI, and really every possible spot a random snippet of the projected quality and cheapness of testing can be made (which is nearly in every page of the UI and repo for some bizarre reason) are very different.

The observatory testing situations and representations of how humans move throughout the simulated environment are extremely impressive and look ready to release, but then you actually connect the ESPs, which I set up in a triangle formation around the room and extended area of a house and see random flickering and glitchy movements with teleporting wireframes that have no limb, gesture, or position approximation whatsoever, forget the 17 "bone" tracking they claim.

I honestly can't even understand what this is supposed to represent since the router in the example is in a fixed position that doesn't reflect its relative position in the real world and the hardware nodes are omitted so there's no possibility of debugging or testing claims empirically or visually.

I think it's a cool project and in its current state should be left as a simulation or educational piece explaining the usages of this CSI technology and the possibilities it allows, but the actual code implementation and data interpretation is abysmal at best. The devs released a model to use with the data to improve prediction but I combed through documentation and Issues on Github and even put an agent up to the task of trying to figure out how to load one of their models and my Gemini instance actually just gave up after trying many different ways and realizing that the rvf files that are required are either 1. impossible to generate with tools given or 2. are so incredibly convoluted to create that no sane user could get it to work without handholding with the main dev.

I don't even understand how the v1 model "honesty issue" can even happen. Any developer, student, or even non-technical person remotely interested in the AI space could look at the 90 something% accuracy claim and take it with extreme levels of caution as that seems absurd for the setup described by the devs and training data used, but somehow it was okayed, pushed, and paraded as a victory of the project. This debacle makes me skeptical of v2, but who knows if it even exists since there's no way to figure out how to use it.

The most amusing part is that, as another commenter mentioned, the hardware "Real time CSI Data" values for phase and amplitude keep changing despite me literally unplugging all hardware and the Raspi running the docker instance. If the devs are willing to lie about a literal metric that the software is anyway supposed to be detecting, how can we verify any of the claims they provide? It's sad that even the people honestly trying to help the dev in Issues get met with Claude Code slop responses that follow the standard AI coding "yes, you're right so here's what we'll do ___" to anything, and a brief 2 liner with no meaningful improvements or takeaways in many scenarios.

Strip the AI slop from the project and you have something cool and fun to play around with and learn from, but by no means is this a prod release worthy of any attention or merit in terms of what the CMU team has researched on. I cannot even believe in good faith that the 86k stars are legitimate after testing the tool and trying the arduous process of setting up everything, but I digress.