Add Python 3.8

[FArekkusu]

• Name: Python
• Version: 3.8
• Release Note/Changelog: https://docs.python.org/3/whatsnew/3.8.html

I've skimmed through the list of deprecated features in Python 3.7 and 3.8, and I didn't notice any changes that would break Python 3-compatible katas. On another note, I think Python 3.4.3 could be removed as well considering it lacks features added with the newer language versions, and everything that works in version 3.4.3 already works in 3.6.0.

---

👍 reaction might help.

[kazk]

I just created List of Python Kata to Update to Python 3 after running a script to enable Python 3 if it's compatible. I've also appended a list of kata that has 3.4, but not 3.6. I'll remove 3.4 after those are updated.

I think some are considered incompatible because it timeouts (#757) which I still don't know the cause. I might experiment with using other images instead of the official one and compare.

[FArekkusu]

Is it possible to replace all xrange instances with range using some automatic script in the Python kata versions? Quite often this is the only thing disabling Python 3.6, and fixing at least 100-200 katas doing this manually will be way too tedious.

[kazk]

Yeah, it should possible to replace all, validate against tests, then store the updated code. But I'll need to write some code to wrap this logic.

For now, I'll update the list and separate the kata with xrange so you can focus on others. Looks like there's 237 with xrange.
Thanks for you help.

[Steffan153]

@kazk All the 3.4.3 kata are updated. Can it be removed now?

[kazk]

@Steffan153 Awesome. I'll remove it soon.

[kazk]

@Steffan153 I found some more kata having only 2.7 and 3.4. See the wiki. Some of them are timing out because of #757 which still persists with Python 3.8 :(

[Steffan153]

Okay, those are done now.

[kazk]

BTW, what's up with the Python 3.4.3? I finished all of those kata.
https://github.com/codewars/codewars-runner-cli/issues/802#issuecomment-617967213

Yes, thanks everyone for helping. I finished the remaining ones on the wiki and updated the page.

Last night, I ran a script to update the default language version of the kata (this script runs any kata with outdated version set to the default and updates the default to the newest valid one), hoping to have all set to 3.6. But this surfaced another set of kata that timeouts because of https://github.com/codewars/codewars-runner-cli/issues/757 . It seems like Python 2.7/3.4 are slightly faster and now 13 kata doesn't have Python 3.6 enabled (one of them timed out in all of them).

I'm debating if I should just update the language versions even it timeouts so I can get rid of 2.7 and 3.4 on Codewars.

I briefly experimented with seccomp profiles from the past Docker versions (https://github.com/codewars/codewars-runner-cli/issues/757#issuecomment-602983343) and found that it doesn't help with performance. So it seems like it's caused by the increased overhead of seccomp in general and not because of a new rule added to the profile. And the only thing the runner can do to speed up Python is to disable it completely and we're not doing that for security reasons.

Have anyone tried using numpy for random tests?

[FArekkusu]

I think we can ignore the issue with Docker affecting the runtime and migrate to Python 3.8 as soon as the remaining katas are updated. numpy is a lot faster with generating numbers, and if some performance-orieted kata requires generating huge inputs, it'd be easier to replace the RNG (or at least lower the input size to some extent) than keep using Python 3.6 and wait (forever?) until everything goes back to normal.

Have anyone tried using numpy for random tests?

I haven't used numpy in any katas myself, but in this translation generating 100 lists consisting of 80k ones/zeroes takes:

• ~3180 ms using random.choices
• ~1350 ms using random.random
• ~180 ms using numpy.random.randint

(Not sure what's the point of the question, considering the whole numpy library is parallelized C code which is obviously lightning fast.)

[kazk]

Not sure what's the point of the question

I just wanted to confirm what the article said (https://github.com/codewars/codewars-runner-cli/issues/757#issuecomment-602983343) was true about seccomp not affecting it much. It should be running close to the native speed. I knew it's obviously faster. But the difference should be more significant when compared inside a container.

[Voileexperiments]

@kazk I can clean up those katas under an hour or two, but currently I'm blocked by these katas because they can't be edited (I already cleaned up those that I can). Is it possible to not wait for a week before they're open for editing?

(For the record, the cause of these katas are #757 for 20% of them, and raulbc777 having very crappy reference solutions for the other 80%, which neither is too hard to fix.)

[kazk]

@Voileexperiments Thanks. It's currently not possible, but I'll add a way to temporarily allow certain users to fix things without waiting. I should be able to deploy that tonight or tomorrow.

[Steffan153]

Since there's no issue about PHP 7.4, is it possible to run a script to add <?php automatically?

[kazk]

@Steffan153 I can do that once all of them are PHP 7.4 compatible. Adding <?php will make it incompatible with PHP 7.0 on Codewars because 7.0 concatenates the submitted files.

[kazk]

@FArekkusu @Voileexperiments @Steffan153 I just gave you guys temporary privilege to fix kata without waiting. You should be able to edit the kata you haven't contributed before. Please let me know if you still can't edit and don't abuse the power. Also, this is experimental and I may revoke it anytime.

[Voileexperiments]

@kazk All Python katas are updated 😉

[Steffan153]

I see that Voile also finished the Nim kata.
Anyway, thanks. I'm working on C++ kata.

[Steffan153]

@kazk C, Coq, and Julia old versions can be removed now.

There's 2 C++ kata left, which should probably be retired, see the wiki: https://github.com/Codewars/codewars.com/wiki/List-of-Cpp-Kata-to-Update-to-Cpp17

[Blind4Basics]

Note: updating python to 3.8 will make almost every code golf kata a f... lot easier. => either we'll need to rewrite them all, changing the char constraint, or we'll need to rewrite them all ( x) ) to ensure only 3.6 is used (if it stays available after adding 3.8)

EDIT: this problem will appear again in the future, I guess... So I'm wondering: would it be possible to have several versions of python, some allowed only in previoulsy created katas and those not having access to newer versions published afterward? That would surely be the best way to get rid of the constant update/rewrite of ultra specific stuff like the "one line task" katas.
I guess it'd be a pain to setup, tho... :/

[FArekkusu]

That would surely be the best way to get rid of the constant update/rewrite of ultra specific stuff like the "one line task" katas

Except everything will have to be rewritten when Python 3.6 will be deprecated as well. You can't support a language version forever only to avoid adjusting some katas' requirements.

[Blind4Basics]

yeah, sure. Just saying that it's normal to have to rewrite some things, but for this kind of katas, the constraints are extremely higly related to the actual language version. So...

[kazk]

So I'm wondering: would it be possible to have several versions of python, some allowed only in previoulsy created katas and those not having access to newer versions published afterward?

That's been already possible. Just make language version a requirement.

import sys
# Only allow Python < 3.8
assert sys.version_info < (3, 8)

[Steffan153]

But if the plan is to delete the old versions, then that won't work - the kata would have to be deleted.
I think the reason he said that is because it sounded like we were planning on removing 3.6.

[kazk]

Yeah, we can retire the language from the kata when it's no longer relevant. If it's the only language, then we can retire the kata as well.

I need to make retiring a language possible. If I remember correctly, deleting a language will delete the solutions list too.

[Steffan153]

So you mean by retiring that the solutions for that lang will still be available, but just not completeable?

[kazk]

Yeah, but feel free to let me know if you have other ideas. I'm open to suggestions except for keeping the old language versions forever.

[kazk]

Deployed.

Python 3.8 have some changes.

• Test framework should be imported explicitly. If not, import codewars_test as test is prepended for backwards compatibility. But it's no longer aliased to Test. If you'd like to continue using Test, add import codewars_test as Test.
• Preloaded code should be imported explicitly. If not, from preloaded import * is prepended to the solution and the tests for backwards compatibility.
• Preloaded code must explicitly import the test framework if necessary.
• Solution should be imported explicitly. If not, from solution import * is prepended to the tests for backwards compatibility.
• /home/codewarrior/solution.txt is no longer used. Read /workspace/solution.txt instead. Older language version have /home/codewarrior symlinked to /workspace, so /workspace/solution.txt will work for those as well.
• Packages were updated to the latest. I'll update the documentation soon.

from preloaded import something # If preloaded exists
import codewars_test as test
from solution import add

@test.describe("example")
def test_group():
    @test.it("test case")
    def test_add():
        test.assert_equals(add(1, 1), 2)

I'm planning to run a script to update compatible kata automatically sometime soon and will create a list of incompatible kata like usual.

[FArekkusu]

@kazk where's the solution located now? Doing open("/home/codewarrior/solution.txt", "r") or open("/home/codewarrior/solution.py", "r") results in FileNotFoundError: [Errno 2] No such file or directory.

[error256]

@FArekkusu As far as I know, it's /workspace/solution.txt for all versions of all languages. In older versions /workspace is a symlink to /home/codewarrior, so it should work there too.

[hobovsky]

To quote kazk from another thread:

It's always at /workspace/solution.txt for any language. Older language versions have /home/codewarrior symlinked to /workspace so /home/codewarrior/solution.txt works on those, but should be updated to /workspace/solution.txt.

[kazk]

Sorry, I forgot to mention that, updated. There's 32 kata using /home/codewarrior/solution.txt so I'll either update them myself or include it in the list of kata to update with a note.

[FArekkusu]

@kazk this kata doesn't import the solution and the test framework, but Python 3.8 is available. Did something change?

Edit: are from solution import * and import codewars_test as test prepended automatically everywhere? I thought this would affect old katas only.

[Blind4Basics]

what do you mean? I don't see any problem on my side :o

[Steffan153]

@FArekkusu Sure.

Test framework should be imported explicitly. If not, import codewars_test as test is prepended for backwards compatibility.

Solution should be imported explicitly. If not, from solution import * is prepended to the tests for backwards compatibility.

(also with the preloaded)

[kazk]

I'm assuming you guys figured out, but let me know if there's anything that's still unclear.

I thought this would affect old katas only.

Nothing changed for the older language versions. The older versions have been doing similar hack/magic and will continue to. I won't introduce breaking changes to existing language versions unless the language itself is in beta.

From Python 3.8, you can import explicitly to disable those magic. I don't like the runner doing anything special (adding code you didn't write, concatenating, etc), so I will slowly deprecate them as we add new language versions.

[FArekkusu]

Is it possible to perform an automatic update of all katas and replace Test with test (limiting it to the testing framework functions, ofc)? There're hundreds of katas which are incompatible with Python 3.8 because of this change.

[kazk]

I'm planning to add a way to make small changes to tests safely. Basically just modify, run tests, then revert if anything unexpected happens. I'm currently little busy, so I can't give any timelines yet.

For Python, it should be safe if we used full function names (Test.assert_equals instead of just Test) to replace, so I might just do it.