HMP and HMI support #75
Comments
|
Looking at specs I have for HM? it could be interesting since there is also loop instructions which are not part of the standard midi format. However since MIDI does support non-standard controllers we could add support for apogees' loop controllers and convert the HM? loop instruction to use that (if we can work out the amount of times we want to loop since HM? just has a marker that shows where to loop back to once the end is reached but nothing saying the amount of times to loop) Specs I have also don't have the format of the track data just the HMI-MIDIP013195 header up to the track data, might have to hexedit a file of this type to decipher the track format.
|
|
I have some HMP files (original format) for testing, hex dump shows the specs I have are only partially correct. If you can find some HMI files I would appreciate that. The HMP channel data seems to be partially midi instructions, still deciphering it. |
|
http://www.vgmpf.com/Wiki/index.php?title=Formats |
|
http://www.descent2.com/ddn/specs/hmp/ <-- more information |
|
File format description by WHS (whs@xs4all.nl) dd 28-2-1999. The internals section below by Dan Wentz (http://descent2.com/ddn/specs/hmp.html) The HMP music format: (determined by hexdumping many hmp's)
1. ==== ORIGINAL FORMAT (as used in descent) ======================
Header 0x00 - 0x07: "HMIMIDIP"
0x08 - 0x1F: 0's
0x20 - 0x23: LSB int32, file length.
0x24 - 0x2F: ?? (in all examined hmp's: zero's)
0x30 - 0x33: LSB int32, number of chunks.
0x34 - 0x37: ?? looks like an LSB number (which is one of 78,
C0, 180, 1E0 in all hmp's I've seen)
0x38 - 0x3B: LSB int32, midi division (with tempo = 1.0 s/beat (i.e. 60 beats per minute))
0x3C - 0x3F: LSB int32, duration of the song (in seconds)
0x40 - 0x307: ?????? looks like 4 bytes per instrument
(175 instruments + 3 extra?).
Data 0x308 .. : music chunks.
Chunk format (positions are relative to the start position of the chunk):
0x00 - 0x03: LSB int32, chunk number.
0x04 - 0x07: LSB int32, chunk length (i.e. including chunk header).
0x08 - 0x0B: LSB int32, track number (?)
0x0C .. : MIDI data, which is standard MIDI, except for
variable length encoded numbers, which have the
LSB first (and a byte here means 7 bits), with
bit 7=1 except for the last byte.
2. ==== 31-1-1995 FORMAT (frontier, slipstream 5000) ==============
Header 0x00 - 0x0C: "HMIMIDIP013195"
0x0E - 0x1F: 0's
0x20 - 0x3F: same as old format.
0x40 - 0x387: ?????? looks same as in the old format but with
extra 0's added ..
Data 0x388 .. : music chunks.
Chunk format is the same as in the original format.
3. ==== 15-6-1995 FORMAT (abuse) ==================================
Header 0x00 - 0x11: "HMI-MIDISONG061595"
0x12 - 0x1F: 0's
0x20 - 0x2D: 0's
0x2e - 0xAD: ?? FF's, with numbers 0-9 interspersed.
0xAE - 0xCF: ??
0xD0 - 0xD1: LSB int16, ??
0xD2 - 0xD3: LSB int16, midi division
0xD4 - 0xD7: LSB int32, song duration (in seconds)
0xD8 - 0xDB: LSB int32, ??
0xDC - 0xDF: LSB int32, ??
0xE4 : number of tracks
0x172 - 0x175: LSB int32, offset of first track
0x176 - 0x179: ditto for the second track etc.
Tracks: (positions are relative to the start position of the track):
0x00 - 0x0C: "HMI-MIDITRACK"
0x57 - 0x5A: LSB int32, track header size
0x5B - 0x5E: LSB int32, same.
[track header size] - ... : midi data. Sort of. Not yet deciphered...
The HMQ music format:
Hmp & hmq are really the same format, but hmq is only used for
OPL synthesizers and uses many more (non general midi) drum
instruments which is why hmq's don't sound nicely when playing
them with a standard midi player.
WHS (whs@xs4all.nl)
-------------------------------------------------------------------------------
Descent Developer Network - HMP/HMQ specs
Safe MIDI composition (i.e. hmp's that won't crash the game.. WHS)
1) OK first of all it's good practice to place all non-note midi events
(chorus/reverb/pan/volume/patch/expression etc) before any note events
start. So for example...
[ here was hmp1.gif => converted to text: ]
____________________|1___.___.___.___|2___.___.___.___|3___._
track 1 | * |$
track 2 | * |$
You'd want to first shift the whole song to begin at measure 2. And place
all non-note events (initializing channels) somewhere between the second
beat of the first measure (*) and the first beat of the second measure ($)
where the note data would begin to play.
2) I'm not sure about other conversion utilities, but this is required for
HMI. Track one should be labeled "Loop" and contain only the events
mentioned in the next step.
* All instrument tracks for General midi songs need to be named with
"[G]" preceding whatever instrument name is used (instrument name is
optional but good to do for reference) For example "[G] grand piano."
* FM song's instrument tracks need to start with [F].
* Furthermore, GUS cards can't read instruments unless you include a "U"
So for a safe guard, D2's midi tracks all started with "[GUF]" to cover
all bases.
* Track names may NOT contain a hyphen!
3) Here's the fun part; Loop points. Say your song is 48 measures long, 4:4
time signature and you're using 120 ticks per quarter note. You would need
to insert event messages as followings.
Forgive the crude use of symbology :)
[ here was hmp2.gif => converted to text: ]
____________________|1___.___.___.___|2___.___.___ ... _|47___.___.___.___|48
track 1 LOOP | *|$ | @|#
track 2 [G]piano | ^|$ | &|
track 3 [G]bass | ^|$ | &|
* LOOP TRACK ONLY:
"*" is at 1:4:119, Event message 110 with a value of 0
"@" 47:4:119, 111 set to 0.
"#" 48:1:000, 1 set to 0
* FOR ALL OTHER INSTRUMENT TRACKS:
"^" 1:4:119, Event message 1 with a value of 0
"&" 48:1:000, 1 set to 0
All midi events on Tracks 2.3 etc that occur between Track 1's event
(*)110/0 and (#)1/0 will loop. Note that the alignment of messages need to
be precise for each respective track.
4) Drums always are assigned to Midi channel 10 (duh) :)
5) Voices are always prioritized to the lower midi channels. So long
sustained notes, or melodies that need to be assured to play with full
duration, need to be placed on the lower channels. Bass, etc (anything that
doesn't need to sustain as long) should be placed
higher.
6) Always beware of cluttering midi events. Depending on your sequencer, you
may be able to automate volume/pan etc. Try not to place too many successive
midi events within the song. This will put a higher tax on your cpu and
isn't necessary.
7) It's always a good idea to manually set each patch with a proper event
message. See (1) for best placement.
Written by Dan Wentz / Last updated: 03/31/99 |
|
https://dl.dropboxusercontent.com/u/396161/hmi_formats.zip |
|
Reading the specs + hex dump of HMP = hair pulling experience ... going to have to write a validator to try to decipher what is going on between events ... |
|
The only HMP in the hmi_formats.zip is a file which does not appear to be a valid hmp file ... not even started with HMIMIDIP ... There is a list of hmi files but I have yet to look at them ... the other zips contain midi version of files but not hm* versions :( Also there appears to be source, please don't taint yourself by looking at it as I don't see any licence info in that zip |
|
Re-using the code not, but you are allowed to read and reimplement the code. However, I've had success in convincing people to re-license their work in the past. :) I know there is at least one that is LGPL, but the rest seem to be GPL or some BSD-3 (like) licenses or none at all. |
|
https://dl.dropboxusercontent.com/u/396161/hmi_formats.zip |
|
https://dl.dropboxusercontent.com/u/396161/daggerfall_soundtrack.tar.bz2 |
|
https://dl.dropboxusercontent.com/u/396161/wr_src0.73.zip |
|
Also don't forget that foo_midi also support HMI and HMP: |
|
Just ripped HMP/HMQ files from the decent2 demo: |
|
Magic Carpet (with a junk headers before HMI part): |
|
Descent: (Demo) |
|
HMP support added to DevTest... see #77 |
|
works well on all the hmp files I had! |
|
You will be happy to know that if all goes well DevTest will be ready to Sent from my iPhone On 7 May 2014, at 7:52 am, Bret Curtis notifications@github.com wrote: works well on all the hmp files I had! — |
|
Can you update the docs we have in the repo under docs/formats/ ? Puh... leeease? :) We can use that as leverage for the DRO2MIDI license change. |
|
I will once I commit to my branch and you guys have tested HMP/HMI with What is scary is I can read midi events in hexdump ... Makes it a lot As I have stated I will be using the validation code as the guide for the Sent from my iPhone On 8 May 2014, at 5:15 pm, Bret Curtis notifications@github.com wrote: Can you update the docs we have in the repo under docs/formats/ ? Puh... We can use that as leverage for the DRO2MIDI license change. — |
|
Throw ya .hmi files at it ... #77 |
|
Everything worked well... HMP from descent1 and 2, and the HMI from daggerfall. However the HMI from Abuse failed. |
|
https://dl.dropboxusercontent.com/u/396161/abuse_intro.log You can find more of these files in the Abuse tarball I posted above. |
|
try again :) |
|
and just committed a fix to remove the flood of debug info :) |
|
Abuse tracks work now. Cheers! |
|
A couple for fixes or slight changes to DevTest. Written basic HMP and HMI specs. They are in the DevTest branch of my fork. |
|
I actually went ahead and just did what I said in the 0.3 branch: see commit faa002d, which adds "WM" prefix to library's private global vars and fuuncs, and if they already are prefixed by "WM_" then it just adds "_" to them. Done this in all library sources for all vars and funcs which aren't static and aren't marked as WM_SYMBOL. Didn't do it in the master branch because it would mess your merge with the DevTest branch. You can do this inn the DevTest branch by yourself, or I can do it for you if you want. |
|
You can do it for me if you like, I won't be making any more changes until I find this bug with the audio. |
|
Done: commit b63cc4e Then let's make it a common rule that any vars and funcs not static and not to be exported should have names begin with "WM". |
|
some how I ended up with a whole lot of conflicts, but no mater, I sorted through them fine thanks to Xcode :) |
|
It has been awhile, just wondering how things are going here? |
|
On 6/19/14, Bret Curtis notifications@github.com wrote:
AFAIK, Chris did it in devtest branch but he also announced that |
|
I haven't seen him on IRC in awhile, so I assume real life caught up with him. ;) @chris: Mind if I take your documents and post them on http://www.shikadi.net/ ? He promised to re-license some of his work that can possibly help us with other formats. |
|
Sorry for the delay. Got caught up with other things. I will attempt to The bug I am encountering it the only thing holding me up at the moment As for docs, wait until I fix this and you guys have tested fully please. While I am updating info, when I do my next commit gauss resampling will Chris. Sent from my iPhone On 19 Jun 2014, at 6:47 pm, Bret Curtis notifications@github.com wrote: I haven't seen him on IRC in awhile, so I assume real life caught up with @chris https://github.com/Chris: Mind if I take your documents and post — |
|
Have you guys been testing this? It is in wildcode-updates branch. Please test and comment. |
|
Hey guys, I saw your project and I wanted to let you know there's another game I am currently working on that uses some form of HMP files: Fatal Racing. |
|
There are different versions of the format: HMI/HMP. If you can, try compiling from our master branch and see if you can play the file back. Or give us a link to the HMP files so that we can give it a go please. :) |
|
Thanks for your quick response. The files have the version 2 header I believe: "HMIMIDIP013195". I compiled from your wildcode-updates branch and it said it was unrecognized. I'll recompile from the master and see how it goes, unless that was the version with hmp support. As to the files themselves, you can download a demo of fatal racing from various sources, or if you would prefer I can send them directly to you. |
|
@chrisisonwildcode You interested in this? @MoffD yeah, just point us in the right direction with a link. :) |
|
Let me know if this link works (I never know with dropbox permissions) |
|
The link works, keep it up for the others to look at. Thanks! :) We'll eyeball it and let you know. |
|
Actually, I just popped it into the hex-editor and found out that each of the hmp files you sent have a 5 byte buffer in the front. That explains why we couldn't read the record, I bet if you trim those bits, it should play just fine. Some of the buffer bytes are: Looks like 00 0F is the end marker for whatever came before "HMIMIDIP013195". Are you sure the files were ripped right? |
|
Odd, they haven't been ripped in any way. Those are directly from the games resource folder. |
|
There is more going on here... so I trimmed up the ingame1.hmp file by five bytes using: and ran our file verifier. It detects the file as indeed HMPv2 but then expects to see 18 zeros after the header... these files do not have that. It isn't a valid HMPv2 file, at least according to our specifications. For example, if we let it through and do the calculations based on the data given us: Testing: /home/bcurtis/Downloads/HMPs/ingame1.fixed.hmp HMPv2 format detected File length: 299665409 Number of chunks: 25233920 Beats per minute: 34725001 Song Time: 365691904 Chunk number: 770346443 Chunk length: 64351028 File too short FAILED: /home/bcurtis/Downloads/HMPs/ingame1.fixed.hmp will not work correctly with WildMIDI Those values don't make any sense. @chrisisonwildcode: What do you think is going on here? For control, here is what D2DEMO looks like (HMPv1): File length: 86569 Number of chunks: 22 Beats per minute: 120 Song Time: 182 Chunk number: 0 Chunk length: 16 Track Number: 0 delta: 30 Meta Event: End Of Track ======================== |
|
Possibly some form of compression? though that seems unlikely due to the fact that other compressed resource files have a *.bm extension, and everything else is uncompressed. |
|
ok, I've taken a look at the 2 of the files that were in the dropbox ... the data they contain is not valid for the format they state as is (taking into account the 5 leading bytes). To verify it wasn't some sort of variation of the hmp file format I looked for MIDI style data which is easily read with a hex editor. Unfortunately the data does not even make sense from a MIDI context. I'm betting these files are compressed/encrypted/modified from hmp files. |
|
How are doing on HMI/HMP support? We good here? Documentation is fixed up? If so, please close this sucker! :) |
|
I'll check docs later today and close once done |
|
Looks all good, will close this one up. |
|
Great! |
|
I am unable to play HMI files I manually extracted from the RESOURCE file for the game 3D Ultra Pinball by Sierra. I think I extracted the data at the correct offsets. I started extracting the songs with the very first bytes in the file being "HMI-MIDISONG061595", and as for the ending of each song file, I wasn't so sure of where the file is supposed to end. In the attachment, you can see the 2 different file cutoffs I tried, one will end just before the next instance of "HMI-MIDISONG061595", and the other song was extracted from the very end of the RESOURCE file (so it will have a slightly different file end). Neither play in this program, but I was able to play them in Foobar2000 using a midi plugin. Is this program having trouble, or did I extract the data wrong? HMIs in question are included. |
Seems to say it is valid but has a lot of notes still on in the end. I just played it back with wildmidi without a problem. They play fine to me, it is about 41s long. Here is the rendering to mp3: I think perhaps the problem lies on your side? Can you play back any other midi's? |
|
I'll have a look tonight, it'll be a nice change from the norm.
…Sent from my iPhone
On 7 Feb 2017, at 1:02 am, Bret Curtis ***@***.***> wrote:
***@***.***:~/Workspace/Private/Wildmidi/build$ ./wildmidi-devtest ~/Downloads/HMI.songs/song.hmi
DevTest for WildMIDI 0.4 - For testing purposes only
Testing: /home/bcurtis/Downloads/HMI.songs/song.hmi
116 notes still on after end of file
Success
Seems to say it is valid but has a lot of notes still on in the end.
I just played it back with wildmidi without a problem. They play fine to me, it is about 41s long.
Playing song.hmi
[Approx 0m 41s Total]
[ ] [100] [ 0m 41s Processed] [100%] /
Shutting down sound output
Here is the rendering to mp3:
https://dl.dropboxusercontent.com/u/396161/song.mp3
I think perhaps the problem lies on your side? Can you play back any other midi's?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.
|
There are 3 versions that need to be supported, but thankfully there are many examples and specifications.
Human Machine Interface (HMI) is a MIDI-like format named after the company that created it, Human Machine Interfaces. The format is derived from the earlier HMP format. The format was used by the Sound Operating System middleware library for audio playback, a competitor of the XMI used in the Miles Sound System.Human Machine Interfaces MIDI P (HMP) is a MIDI-like HMI created by Human Machine Interfaces. The format seems to lack tracking tags, so the format may be something akin to a MIDI 0 file.
The extension is generally .hmp. Descent used the extension .hmq for song variants destined to be played on FM synthesis sound cards.
Technical
There are two existing versions of the HMP format. All HMP files begin with "HMIMIDIP". The oldest version has its signature followed by a null byte; while the revised version, dated from January 31, 1995, has the signature followed by this date (HMIMIDIP013195). A second revision, dated from June 15, 1995, resulted in the HMI format.
This covers music from:
1994-05-06 Magic Carpet (DOS)
1994-??-?? Sensible Golf (DOS)
1995-03-17 Descent (DOS)
1995-08-?? Abuse (DOS)
1995-??-?? The Terminator: Future Shock (DOS)
1996-03-13 Descent II (DOS)
1996-03-13 Descent II (DOS)
1996-08-31 Daggerfall (DOS)
1996-??-?? The Terminator: SkyNET (DOS)
1997-06-30 Carmageddon (DOS)
1997-07-15 X-COM Apocalypse (DOS)
The text was updated successfully, but these errors were encountered: