Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to stop a sequence when certain contion is matched? #445

Open
exaos opened this issue May 28, 2018 · 5 comments
Open

How to stop a sequence when certain contion is matched? #445

exaos opened this issue May 28, 2018 · 5 comments
Labels

Comments

@exaos
Copy link

exaos commented May 28, 2018

kmax-event-file-structure
As the attached picture, a sequence might stop at certain condition, e.g., when encountering some magic or next block is not the same data type. How to depict this kind of conditional reading? How about this:

seq:
  - id: data_block
    type: data_type
    repeat-if: next_is_the_same_type
@GreyCat
Copy link
Member

GreyCat commented May 28, 2018

Just use repeat-until with a block type that will consume both real "event blocks" and "event file footer" as a special terminator block. I have no idea what Ev-1 :: [Parmeter-1] etc stand for, but if that's just several 32-bit integers, then you can always do something like that:

seq:
  # ...
  - id: more_blocks
    type: event_block
    repeat: until
    repeat-until: _.is_terminator
type:
  event_block:
    seq:
      - id: param1
        type: u4
      - id: param2
        type: u4
      - id: params_rest
        type: u4
        repeat: expr
        repeat-expr: _root.ev_size - 2
        if: not is_terminator
    instances:
      is_terminator:
        value: param1 == 0xffffffff and param2 == 0xffffffff

@GreyCat
Copy link
Member

GreyCat commented May 29, 2018

Yeah, right ;) Would fix now.

@exaos exaos closed this as completed May 30, 2018
@exaos
Copy link
Author

exaos commented May 30, 2018

kmax_event.zip
Thanks GreyCat! I prepared a .ksy file, and then generated the python code without error. But the python code will consume all the memory of my PC and halt my PC. Maybe it is the problem of repeat-until. Though I don't know why, I will check it again.

@exaos exaos reopened this May 30, 2018
@webbnh
Copy link

webbnh commented May 30, 2018

@exaos, how big is your data file (in comparison to the amount of memory on your PC)? Your program is going to try to read all of the data into memory....

I ran into this problem in my project, and had various problems with Python and memory consumption. I ended up implementing the repeat: until in my own code, by invoking KS repeatedly until my own end condition, and carefully releasing each iteration's memory before starting the next iteration. This actually works quite well (although I ended up switching from Python to C++, for several reasons).

@exaos
Copy link
Author

exaos commented May 31, 2018

@webbnh
My test data file is less than 1MB. The typical data files are around ~20MB each. And my PC's memory is 16GB. Theoretically, I could read all data from a test file without any problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants