-
Notifications
You must be signed in to change notification settings - Fork 134
Knowledge Nuggets
This page contains snippets of useful knowledge and tips related to Ballistica development that are not big enough to warrant their own page. Feel free to add your own here or upvote existing ones by incrementing the 👍 count next to them (just once per tip please). I will use upvote counts to help organize the tips by usefulness.
- Mypy: Getting a Variable's Type
- Debug-Only Python Code
- Python Type-Checking Boilerplate
- Data Classes are Handy
- Enums are Handy
- Makefile Autocompletion
- Python Cache Files and Release Builds
Is there something you'd like to see here? Add it to this list or increment the 👍 count on an existing one.
- How do I create my own Spaz type? 👍 0
When using Mypy to perform static type-checking on Python code, one of the most basic and important things to be able to do is determine the static type of a variable as Mypy sees it.
Say we have the following code:
value = some_mysterious_function()How do we know what type Mypy thinks 'value' is? We could guess this ourself by looking up the definition of some_mysterious_function(), but it is often better to be sure and just ask Mypy directly. We can do that by adding the following special fake function call to our code which is understood by Mypy:
value = some_mysterious_function()
reveal_type(value)Now we can run make mypy in a terminal from the project root, and we should see output such as this:
Running Mypy (incremental)...
path/to/this/python_script.py:45: note: Revealed type is 'builtins.int'
Mypy: fail.
make: *** [mypy] Error 255
Ok, Mypy says 'value' is an int. Good to know! So if we were to add a statement such as value += 'foo', then Mypy would kindly inform us that ints and strings cannot be added. Without type checking, an error such as that might go unnoticed until the next time the code gets run, and who knows when that might be. Hooray for static type checking!
Be sure to remove the reveal_type() line when you are done with it, otherwise the code will error at runtime due to that not being an actual Python function.
👍0
With the Ballistica project it is now easy to switch between debug and release builds of the game. (make prefab-debug vs make prefab-release). Debug builds perform much more error checking than release builds; this makes them very useful during development, with the tradeoff that they can be significantly slower and less efficient than release builds.
You can take advantage of having two build types by adding extra safety checks that run only in debug builds. There are a few ways to do this.
The first is the Python's 'assert' statement. These will be evaluated in debug builds and completely stripped out of release builds, so think of them as 'free'. They are a great way to make sure things are as you expect them to be.
my_thingie = make_a_thingie()
assert my_thingie is not None. # <-- Throws an AssertionError if not True.As a nice side-effect, assert statements are also quite useful for Mypy type-checking. (see Mypy: Getting a Variable's Type)
# Dynamically evaluate a string to create an object (in this case an int).
val = eval('123')
# If we run 'make mypy' from the project root, Mypy will tell
# us that val is 'Any' here because eval() can return anything.
reveal_type(val)
# We expect val to be an int; let's make sure that's the case.
assert isinstance(val, int).
# Because of the above assert statement, Mypy knows val must be an int
# at this point, so 'make mypy' will show 'builtins.int' here.
reveal_type(val). Another way to generate debug-only code is with Python's special __debug__ variable. Code such as this will run in debug builds of the game but will be completely stripped out of release builds, again making it 'free'.
if __debug__:
for node in all_my_nodes:
do_some_expensive_sanity_checks(node)With any code such as this, it is extremely important that no actual logic be affected as a side-effect of these expressions since they will not be run at all in release builds.
👍0
You may notice certain things at the top of most Ballistica Python modules; namely this code:
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
import xxxThe following is an explanation of why that code exists:
Ballistica uses Mypy for static typing. Static typing lets us catch a large number of issues in our code without having to run it. But for this to work, we need to add type annotations to our functions and variables.
# So instead of this:
def get_next_number(input):
return input + 1
# We need this:
def get_next_number(input: int) -> int:
return input + 1Pretty simple so far. However, sometimes we can't annotate things because the names we want to use don't exist yet. Consider the following where we want to create a 'my_thingie' variable that can either point to None or a Thingie instance:
from typing import Optional
my_thingie: Optional[Thingie] = None
class Thingie:
passIf we run this code, it will fail, saying 'Thingie' is not defined on the line where we create my_thingie. This is true; it won't exist until Python executes the code 2 lines down from it that creates the class. And yes, in this case we could simply move the class definition above the variable definition, but ignore that for the sake of this exercise.
Python type annotations work around problems like this by allowing forward declarations using strings:
from typing import Optional
my_thingie: "Optional[Thingie]" = None
class Thingie:
passThis code will work, but it makes the annotation look ugly. Luckily, because Ballistica uses Python 3.7+, we can make this cleaner by using Postponed Evaluation of Annotations. This essentially turns all annotations into strings under-the-hood, so we can write more natural looking code but still forward-declare things. This also makes things more efficient since Python doesn't have to evaluate complex annotation code such as List[Dict[str, Optional[Thingie]] at runtime; it simply sees that whole thing as a string. This behavior will be default in a future version of Python, but for now we have to use a __future__ import to get it.
So now our code looks like:
from __future__ import annotations
from typing import Optional
my_thingie: Optional[Thingie] = None
class Thingie:
passThis code runs and isn't too ugly (aside from the __future__ bit which we will eventually be able to get rid of). But there is one final issue. Running make check on this code will give us an error, with Pylint telling us that the imported 'Optional' is unused here. This is technically true because my_thingie's annotation is really just a string now; no actual runtime code is using Optional. However it is needed by the type-checker; if we remove that line and run make check again, then Mypy type checking will fail saying Optional is not defined. So how do we make everyone happy here?
The answer is to define such things only for the type checker. We can do that using a special TYPE_CHECKING constant provided by the typing module. So the final form of our code is this:
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from typing import Optional
my_thingie: Optional[Thingie] = None
class Thingie:
passSo now everyone is happy; we're not being wasteful at runtime by importing things we don't use, but the type checker is still able to find everything it needs. This leaves us with a few lines of boilerplate code per module, but it gives us a good type-checking foundation. With this setup we can isolate imports and other code that is only used for type-checking, keep forward-declarations clean and readable, and minimize the impact of type-checking on runtime performance.
👍0
Python3.7 introduced data classes, so we now have access to these in Ballistica. These work very well with our type-checking setup, allowing us to easily define little bits of data that get passed around. We could also write full classes for these purposes, but that is often overkill and would require extra boilerplate code.
Here is how we might have done some things before, using simple dicts and values:
# Hmm; let's keep track of some spawn points for our mini-game or whatnot...
spawn_points = []
spawn_points.append({'position':(0.0, 0.0, 0.0),
'team':0})
spawn_points.append({'position':(0.0, 0.0, 0.0),
'team':1,
'color':(1.0, 0.0, 0.0)})
# That works, but it can be easy to introduce subtle errors that
# won't be noticed until runtime:
# Oops; wrong name for the dict entry.
spawn_something_at_position(spawn_points[0]['pos'])
# Oops; our first spawn point doesn't contain 'color'; we'd need to
# check for that case and substitute a fallback value here.
flash_at_position(spawn_points[0]['position'],
color=spawn_points[0]['color'])Using data classes, we can write this code in a much safer manner:
from dataclasses import dataclass
from typing import Tuple, List
@dataclass
class SpawnPoint:
position: Tuple[float, float, float]
team: int
color: Tuple[float, float, float] = (1.0, 0.0, 0.0)
spawn_points: List[SpawnPoint] = []
spawn_points.append(SpawnPoint(position=(0.0, 0.0, 0.0), team=0)
# Now running `make check` will catch all sorts of errors for us without
# having to run the code:
# Error: required 'team' arg not provided.
spawn_points.append(SpawnPoint(position=(0.0, 0.0, 0.0))
# Error: SpawnPoint has no 'pos' attr.
spawn_something_at_position(spawn_points[0].pos)
# And this WILL work now because SpawnPoint gets a 'color' value
# by default even if it wasn't passed in the constructor.
flash_at_position(spawn_points[0].position,
color=spawn_points[0].color)Hopefully this shows how much safer code is when defined in this way. It makes it possible to add, remove, or rename attributes from SpawnPoint later and instantly see exactly which places in the code need to be changed.
👍0
Python 3.4 introduced enumerations, so we now have access to these in Ballistica. These can work well when paired with our type-checking setup, allowing us to write safer code.
Consider the following situation: we might want to write a mini-game that involves spawning bombs at particular places in a map. In the old system we may have written something like this:
def spawn_bomb(where: str) -> None:
if where == 'top_left':
pos = (-10, 0, 10)
elif where == 'top_right':
pos = (10, 0, 10)
elif where == 'bottom_left':
pos = (-10, 0, -10)
elif where == 'bottom_right':
pos = (-10, 0, 10)
else:
raise ValueError(f'Invalid where: {where}')
do_spawn_bomb_at_position(pos)
# This first line will work, but the second will error at runtime.
spawn_bomb('top_left')
spawn_bomb('center_left') # OOPS; not a valid value.That works, but if we want to modify our mini-game with a different set of 'where' options then we have to carefully go through any code that calls spawn_bomb() to make sure none are passing no-longer-valid string values. This is exactly the sort of thing that type-checking can do for us.
Rewriting this to be type-safe using enums looks like:
from enum import Enum
class SpawnPos(Enum):
TOP_LEFT = 'tl'
TOP_RIGHT = 'tr'
BOTTOM_LEFT = 'bl'
BOTTOM_RIGHT = 'br'
def spawn_bomb(where: SpawnPos) -> None:
if where is SpawnPos.TOP_LEFT:
pos = (-10, 0, 10)
elif where is SpawnPos.TOP_RIGHT:
pos = (10, 0, 10)
elif where is SpawnPos.BOTTOM_LEFT:
pos = (-10, 0, -10)
elif where is SpawnPos.BOTTOM_RIGHT:
pos = (-10, 0, 10)
else:
raise ValueError(f'Invalid where: {where}')
do_spawn_bomb_at_position(pos)
# This time around, the error on the second line gets picked up
# by type-checking; no need to run the code.
spawn_bomb(SpawnPos.TOP_LEFT)
spawn_bomb(SpawnPos.CENTER_LEFT) # OOPS; not a valid value.Now make check should ensure that all calls to spawn_bomb() are getting passed valid SpawnPos objects, so if we add or remove values from SpawnPos we can instantly see which code needs to be updated.
The values associated with enums can have basic types such as int, str, etc. and can be useful when storing the enum as JSON, passing it over the network, etc.
enumval = SpawnPos.TOP_LEFT # This is an enum object.
rawval = spawnpos.value # This is the string value 'tl'.
enumval2 = SpawnPos(rawval) # Create SpawnPos.TOP_LEFT from 'tl'.See the enumerations docs for more tricks.
👍0
Ballistica's main Makefile is set up to list info about its available targets when you simply type make or make help.
However, it can also be handy to set up your shell to allow autocompleting target names. If you are using zsh (the default on Mac as of 10.15 Catalina), you can add this to your .zshrc file to enable it:
autoload -U compinit
compinit
Once you restart your shell, you should be able to type something like make prefab- and hit tab to see a list of all matching targets.
ericf@MacBook-Fro ballistica % make prefab-
prefab-debug prefab-mac-release
prefab-debug-build prefab-mac-release-build
prefab-linux-debug prefab-release
prefab-linux-debug-build prefab-release-build
prefab-linux-release prefab-windows-debug
prefab-linux-release-build prefab-windows-debug-build
prefab-mac-debug prefab-windows-release
prefab-mac-debug-build prefab-windows-release-build
The same functionality should be possible with other shells such as bash, but I will leave that as an exercise for the user (feel free to expand this tip).
👍0
TLDR: In some specific cases you may need to blow away .pyc files if you are editing .py files.
When Python imports a module, it attempts to first create an intermediate 'compiled' .pyc file from the raw .py script file. This compiled version can then be reused to import the module more efficiently the next time it is used. In old 2.x versions of Python these .pyc files were created alongside the .py files, and in modern 3.x versions they are placed in a __pycache__ directory alongside the .py files.
Traditionally, Python either recreates a .pyc file or ignores it whenever the timestamp on the corresponding .py file differs from it. This generally does the right thing as long as Python has write access to the .pyc file, keeping the .pyc up to date whenever the .py changes. However, in cases such as game distributions, Python might not have write access to the .pyc files, and bundled .pyc files might end up with timestamps that differ from their .py files as a result of the install process. The combination of these can lead to inefficiencies as Python repeatedly tries (and fails) to recreate these .pyc files. See PEP 552 for more details.
Ballistica now avoids this problem by creating optimized 'unchecked hash' .pyc files as part of its build process. 'Release' builds of the game will look for .opt-1.pyc files for any loaded module, and if one is found, it will always be used, even if the corresponding .py file is subsequently changed. This means these .opt-1.pyc files must be explicitly updated instead of relying on Python to do so.
You don't need to worry about this if you are using Ballistica Makefile targets such as make prefab-release; .opt-1.pyc files will be automatically regenerated as part of these builds. But if you are hacking on scripts directly in an installed copy of the game, be aware that you will need to blow away the script's existing .opt-1.pyc file or your edits will never be seen by the game. (though you should only need to do this once; any new .opt-1.pyc files written by the game should be timestamp based)
Note that this only applies to the release build of the game. Debug builds still use plain old timestamp-based .pyc files and will continue to exhibit the classic behavior.
👍0
ballistica.net • discord • blog • support