Rework PyBeeb into a reusable system, with host interfaces #5
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The files appear to be formatted for DOS line endings, and had trailing
spaces on some lines. These have now been stripped with:
for i in *.py */*.py ; do
echo $i ; sed -E -i '' 's/ *\r?$//g' $i
done
There's now an interface that's similar in style to the Unicorn interface. The intention is that whilst this won't be a drop in for Unicorn, the expectations that you might have from Unicorn, such as the means of accessing registers and memory, and executing code, are similar.
In order to be reusable in other systems, it is better if the whole emulation system be in a package, rather than at the top level of the namespace. This change moves everything into a `pybeeb` package.
The tests hadn't been updated to run in the new packaged structure.
The PyBeeb, PyBeebU and test files are now executable under unix systems and given shebang lines.
The execution hook was happening before the start of the decode because there wasn't an easy way to inject after it. This meant that it was not possible to report the instruction length. It might not be a big problem, but it did mean that we couldn't call the hook as easily. We now call the hook with the right details by having a class that inherits from the Dispatcher do the hook calls.
The last address in the memory map regions would not be supplied - the check for whether the memory was in the mapped object was not allowing for the inclusive end address of the mapping. This meant that the IRQ entry point (which is used by BRK) would be called with the high address byte being returned from the standard memory, which was 0. Instead of calling &DC1C, it was calling &001C. We now check the memory region inclusively.
The instruction exceptions are now standarised to use an Exception for the instructions, rather than a BaseException. The undefined instructions are also reported as exceptions now, as well.
The __repr__ for Registers now includes their values.
The readBytes and writeBytes interfaces hadn't stopped when they reached the end of the requested region because of a failure to account of the end of mapped regions.
The register writes in the PyBeebicorn interface were always being AND'd with 255, even for the PC. The value stored is checked in each writer function now. The execution hooks are now able to reset the PC value when they are run, preventing the execution of the instruction that was decoded.
The BRK now returns the address to call, and reset has a variable instead of just calling a magic address.
The RDCH entry point is replaced by a key reader from the console. This uses the console code that I wrote for RISC OS Pyromaniac, modified to be independant on Pyro. This allows the RDCH entry point to be modified to handle the key input. It's not great because we lose *Exec handling, but since we don't have a filesystem or any other interfaces, that's not a big deal right now.
The timeout value of None means 'wait forever', but it wasn't actually working because it had never been tested. We now check this more carefully.
A debug had been left in the PyBeebicorn interface when the PC was set.
If there's no input, we could return None. This might happen if there had been input but no characters were actually read, or the timeout occurred.
We can now delete characters in the character input handler, and they will be output properly.
The memory access hooks allow us to report when reads or writes happen to memory. They may be problematic at the moment as the store operations appear to read the values from the memory as well, before they then go and store to the locations.
The RDCH function was not reading the escape key properly, because we weren't setting the C flag. It also appears that unless you set the error condition in zero-page, the system won't report the error properly.
I had originally cut down the console.py to be a limited, POSIX only, version of the console.py implementation. However, there's no reason why this cannot be used on Windows so I may as well include the Windows version. It should work reasonably well, but it's not as polished as the POSIX version.
There is now a simple disassembler (more than the original one) in the PyBeebicorn code, which can disassemble the code as it is executed. This can be used in a hook to trace only certain sections of the code.
We can now exercise the code in some tests that can be run in a Makefile. Only the coverage run is explicitly supported right now, and it'll fail if the tests fail, so it's a combined coverage and test reporter. We mix the unit tests and the integration tests at the moment, and the output from the integration test isn't checked for regressions.
There is a simple workflow here to run the coverage and tests on push. We're only running under Python 2, still, but it should be sufficient to make it work for us.
The PyBeepicorn example program now has some more abstracted interface entry points. These are set up in such a way that we can use them without (in theory) having the knowledge of the underlying system. They should be easy to extend for the other APIs.
We now have a very simple OSCLI handler so that we can trap *QUIT and exit the emulator. If we don't recognise the command it is passed to the rest of the handler system.
We now have classes which are hookable so that we can replace the OSBYTE and OSWORD implementations. This allows us to provide some interfaces to the host system which override the BBC implementation.
The host interfaces have now moved into a package so that they can be used more generically if people want. Specific routines that interface with the host are separate from the base implementations so that they don't have to be the way that you communicate with the system.
The filesystem interfaces OSFILE, OSARGS, OSBPUT, OSBGET, OSFIND have been given base implementations which dispatch to different routines. They're not especially well suited to being overridden because each class is independant. Ideally we'd allow a common interface to be provided for the filesystem, which those classes could access.
The base OS interfaces were duplicating some behaviour and not as common as you might like. The interfaces are a little easier to update now. It's not clear that they're actually going to make for a better general experience but they will allow some more flexibility. Also a few typos fixed.
OSGBPB's byte operations are now implemented in a base class. The other GBPB operations that handle the media name, and directory enumeration are not yet implemented.
When the timeout is absent, we weren't handling the cursor key decoding. This meant that they came out as escaped VT codes instead of BBC style codes.
Most of the operations have now been updated so that they work with Python 3, allowing us to run the commands safely with Python 3.9 at least. The code still works with Python 2.7 as well.
We now have the dispatcher handling OSGBPB calls for directory names and the reading of filenames.
The OSFSC base implementation is now provided.
We now have a host filesystem module, fsbbc.py, which provides a BBC-like interface to the host filesystem. It's not tested beyond being run through a few simple exercises, but the name translation is based on code from RISC OS Pyromaniac so it should have the right style of thing in it. The OSFILE load and OSFILE save operations are implemented and these allow BASIC to load files and save them.
We now have the read/write file information and their variants, and the delete operation, implemented in the host filesystem version of OSFILE.
OSFIND, OSBGET and OSBPUT are now implemented. This should hopefully do the right thing for us, although they're not well tested.
The data mode for ADC indexed Y access was listed as `imy` instead of `iny`.
We now implement some of the stream handling in OSARGS, allowing us to manipulate the pointer.
The .pyc files can be cleaned away to make them get rebuilt. Useful if we're changing content.
The hosttty.py file is there to operate only on the TTY, so we might as well name it as such.
Reading a key within a time limit is now supported by the INKEY OSBYTE call.
The EOF check now works, either when called through OSFSC or OSBYTE.
Two hex digits in the code address for OSARGS had been transposed, meaning it was never called. And the EXT handler hadn't assigned the right variable.
Buffering and *Exec are handled as part of OSRDCH, but we need to process the routing a little later, after all the code which retrieves from the file, and checks the buffer. That way, if there is buffered data, we'll use it.
The PyBeebicorn interface is now a main interface, with the name Emulation, instead of PyBeebicorn. This allows it to be passed around more freely. As such, instead of referring throug the regs and memory objects which are passed in, we now pass through the emulator obejct `pb`.
We now have a MOS interface that allows us to call 6502 code which is necessary to write characters to the display. And we use this with the *CAT interface. We also have a CWD in the filesystem now.
We now support *DIR properly, and we have a number of fixes for command line uses.
Python 3 support had been broken when the CLI handling had been reworked.
The documentation has been updated to reflect how the code has changed to focus on the new hooking system and the ability to interact with the host system. The old code has been moved to PyBeebMinimal.py as part of this.
Coverage runs of the Memory tests hadn't been working at all. They were not invoking the unittest on the module. We now correctly invoke that and get better coverage because of it. The coverage had been confused on MacOS because PyBeeb.py and pybeeb/__init__.py are effectively the same module, so we were getting other references to modules that weren't really there. To avoid this problem, and to make it easier to understand the difference between the tool and the package, the command line program has been renamed to `RunBeeb.py`.
The filesystem interfaces should now be reporting the filenames properly to the BBC and back out again to the host filesystem, when used in both Python 2 and Python 3.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This is a large-scale reworking of the PyBeeb sources to make it into a system that can be plugged into other projects. The intention has been to make it easier to work with, be hooked to provide support for debugging functions and be able to be used for small exercises.
Significant changes
pybeeb, which should be able to be used as a library to implement the 6502 system alone, or a 6502 with BBC micro interfaces, as required.pybeebprovides BBC MOS abstractions for interfaces, allowing the MOS to communicate with the host system. These have been written with minimal knowledge of the innards of the PyBeeb system, in the hope that they will be retargetable to other emulators (a bit of an aspiration, but it'd be nice if other Python 6502 systems could use these)..INFfile support could be added, to allow archives to be read more easily.Implemented MOS interfaces
OSWRCHcan write characters to the TTY.OSRDCHcan read characters from the TTY.OSWORD 0can read lines from the TTY.OSBYTE &7Fcan check for EOF.OSBYTE &81can read a character with a time limit.OSBYTE 0can output a different version message.OSFILEsupports loading, saving, read and writing file information and deleting files.OSFINDsupports opening and closing files (but doesn't support closing all files).OSBGETandOSBPUTsupport reading and writing streams.OSARGSsupports the file and extent operations, but not the flush or CLI args.OSFSCsupports EOF and*Catbut none of the other operations.OSCLIsupports*Dirand*Quit.Other operations are not supported, including
OSGBPB.Example usage
The above is available as a video: https://asciinema.org/a/628615