docs(frontend): adding a use-case for Levenshtein distance

[bcm-at-zama]

Adding a use-case for Levenshtein distance

closes #https://github.com/zama-ai/concrete-internal/issues/750

[bcm-at-zama]

Works quite well but fail from time to time:

Computations in FHE

    Computing Levenshtein between strings '' and '' - OK in 0.01 seconds
    Computing Levenshtein between strings '' and 'a' - OK in 0.01 seconds
    Computing Levenshtein between strings 'b' and '' - OK in 0.01 seconds
    Computing Levenshtein between strings 'a' and 'a' - OK in 10.76 seconds
    Computing Levenshtein between strings 'a' and 'b' - OK in 11.03 seconds
    Computing Levenshtein between strings '' and '' - OK in 0.01 seconds
    Computing Levenshtein between strings '' and 'a' - OK in 0.01 seconds
    Computing Levenshtein between strings '' and 'tv' - OK in 0.01 seconds
    Computing Levenshtein between strings '' and 'gag' - OK in 0.01 seconds
    Computing Levenshtein between strings '' and 'wywd' - OK in 0.01 seconds
    Computing Levenshtein between strings '' and 'oezbr' - OK in 0.01 seconds
    Computing Levenshtein between strings '' and 'ctuugd' - OK in 0.01 seconds
    Computing Levenshtein between strings 'n' and '' - OK in 0.01 seconds
    Computing Levenshtein between strings 'o' and 'g' - OK in 11.78 seconds
    Computing Levenshtein between strings 'p' and 'sl' - OK in 21.41 seconds
    Computing Levenshtein between strings 'r' and 'qbd' - OK in 30.90 seconds
    Computing Levenshtein between strings 't' and 'vbej' - OK in 43.54 seconds
    Computing Levenshtein between strings 'b' and 'srvxs' - OK in 52.11 seconds

    Computing Levenshtein between strings 'n' and 'fkuftz' - OK in 63.67 seconds
    Computing Levenshtein between strings 'ey' and '' - OK in 0.01 seconds
    Computing Levenshtein between strings 'kd' and 'f' - OK in 20.81 seconds
    Computing Levenshtein between strings 'fv' and 'xh' - OK in 60.51 seconds
    Computing Levenshtein between strings 'mv' and 'dnr' - OK in 120.93 seconds
    Computing Levenshtein between strings 'db' and 'msvl' - OK in 199.30 seconds

    Computing Levenshtein between strings 'hn' and 'whoql'Traceback (most recent call last):
  File "frontends/concrete-python/examples/levenshtein_distance/levenshtein_distance.py", line 189, in <module>
    assert l1_fhe == l1_clear, f"    {l1_fhe=} and {l1_clear=} are different"
AssertionError:     l1_fhe=5 and l1_clear=4 are different

I have p_error=10**-8 which is already quite low

[bcm-at-zama]

Blocked by https://github.com/zama-ai/concrete-internal/issues/754, and a bit by https://github.com/zama-ai/concrete-internal/issues/752 and https://github.com/zama-ai/concrete-internal/issues/753

[bcm-at-zama]

Now:

• https://github.com/zama-ai/concrete-internal/issues/754: no more semantic issue (with wires + @rudy-6-4 has a fix for the issue)
• https://github.com/zama-ai/concrete-internal/issues/752: good now
• https://github.com/zama-ai/concrete-internal/issues/753: closed

[bcm-at-zama]

Hey @umut-sahin , would you have a look, to see if we can make it faster, please?

[bcm-at-zama]

Now I have added a system for alphabet: eg, one can do

python frontends/concrete-python/examples/levenshtein_distance/levenshtein_distance.py --autotest --alphabet ACTG --show_mlir

and you'll see it generates random string of A, C, T and G.

[bcm-at-zama]

It's very easy to add a new alphabet

[bcm-at-zama]

And there is a new option, to make perks on all alphabets:

python frontends/concrete-python/examples/levenshtein_distance/levenshtein_distance.py --autoperf

Typical performances for alphabet ACTG, with string of maximal length:

    Computing Levenshtein between strings 'AACG' and 'GATT' - OK in 5.24 seconds
    Computing Levenshtein between strings 'ACCG' and 'GTGG' - OK in 4.71 seconds
    Computing Levenshtein between strings 'TTTC' and 'TTTA' - OK in 4.87 seconds

Typical performances for alphabet string, with string of maximal length:

    Computing Levenshtein between strings 'skon' and 'iisi' - OK in 14.67 seconds
    Computing Levenshtein between strings 'qukm' and 'vufu' - OK in 15.01 seconds
    Computing Levenshtein between strings 'afoe' and 'kbwh' - OK in 14.62 seconds

Typical performances for alphabet STRING, with string of maximal length:

    Computing Levenshtein between strings 'RPJX' and 'MLZU' - OK in 14.33 seconds
    Computing Levenshtein between strings 'GYAQ' and 'IEWC' - OK in 13.73 seconds
    Computing Levenshtein between strings 'GEUC' and 'CLUI' - OK in 15.11 seconds

Typical performances for alphabet StRiNg, with string of maximal length:

    Computing Levenshtein between strings 'oXcM' and 'Igjh' - OK in 30.11 seconds
    Computing Levenshtein between strings 'pgBk' and 'GuOp' - OK in 28.20 seconds
    Computing Levenshtein between strings 'jScn' and 'yRRN' - OK in 30.81 seconds

Successful end

[bcm-at-zama]

What do you think guys? When you like the code, I add a small .md and measure that on AWS

[bcm-at-zama]

Blocked by https://github.com/zama-ai/concrete-internal/issues/794

[bcm-at-zama]

No more blocked by 794

[umut-sahin]

I'd really prefer to have a class for the alphabet with encode, decode, check, ... methods. It'd make the example much clear IMO.

Also you would be able to do:

for alphabet in [Alphabet.lowercase(), Alphabet.uppercase(), Alphabet.dna(), ...]:

Which is less error prone and more clear IMO.

[bcm-at-zama]

I'm not a big fan of OOP, especially when like here, it doesn't seem needed. But if you want really it, I'll do. (And for sure, I have not the right habits for OOP so I am going to make something not idomatic, you'll correct me)

[umut-sahin]

I don't think this is OOP. Sure it has a bit of encapsulation and maybe some abstraction, but there is no inheritance or polymorphism. It's just a way to make alphabets more type safe and understandable.

Would be better to me, wdyt @BourgerieQuentin?

[BourgerieQuentin]

Minor comment feel free to include or not, thanks.