Find file
Fetching contributors…
Cannot retrieve contributors at this time
333 lines (326 sloc) 19.8 KB
dircache_init(): buffer_alloc for each d_name in the opendirs array
* the opendirs dirs array is statically allocated, there's no
reason the d_name in it shouldn't also;
* reason for buffer_alloc() is probably that d_name needs to be a pointer
dircache_load(): first buffer_alloc(0) to determine the start of buffer
* this is in within HAVE_EEPROM_SETTINGS, the first buffer_alloc()
is to check whether the buffer start matches the buffer start
in the dump on disk, if it it is the dump is loaded to memory as-is,
otherwise dircache is fully reinitialized
* this relies on reproducable boots (identical buffer_alloc() calls) and
determininistic buffers from buffer_alloc(). This should work with
buflib also. In any event, it only affects HAVE_EEPROM_SETTINGS (h100?)
and it's probably non-fatal if it doesn't work anymore
*dircache_build(): buffer_alloc()s if the dircache size is known (e.g.
from previous boots), otherwise audiobuf is accessed directly.
* in the first case, the buffer size is known and allocated,
followed by a "transparent build" (i.e. in background);
should be easily replacable with buflib
* in the second case, the buffer size is unknown and a foreground
build is started; this relies on no other thread being able temop to
run for exclusive access; not possible with current buflib
-> need to implement some buflib_get_all() which may possibly
(see also dircache_do_rebuild())
usage of the allocation:
if not buffer_alloc()d, it's only temporarilly used, probably no
concurrent access (CA) issues (that or it would be completely
broken currently)
if buffer_alloc()'d, then the buffer is used throughout runtime to
allocate dircache_entry structs and the d_name member of it (separately,
d_name is allocated as needed); access happens via dircache_root and
offset (dirache_root+dircache_size, etc)
considerations for compaction
dircache.c itself can probably handle compation just fine, since
it's using an offset-to-pointer mechanism already and keeps track
of all sub-allocations (via dircache_entry->next etc)
however, dircache_get_entry_ptr() exports dircache_entrys (and thus d_names)
from the buffer to the caller for unknown time; therefore compaction
of this buffer is not always safe :(
Additionally, a pointer to a dircache_entry is saved in DIR_CACHED,
every DIR_CACHED is from the opendirs array (local to dircache.c)
and they have a busy flag; DIR_CACHED is opaque for callers (so
no direct dircache_entry access) and d_name is copied to separately
allocated storage
-> this perhaps needs a
lock, user supplied buffer to copy the entry or invalidation (need
to analyze the callers more; only tagcache.c and playlist.c so far)
dsp_timestretch_enable(): buffer_alloc() on boot, and only if timestretch
is enabled. size is fixed.
the buffer is used for resamping, depending on timestretch.
resample_buf is toggled between small_resample_buf and big_resample_buf.
It's then used only used in resample(), apparently without yield().
resample() doesn't know if it's a buflib buffer or a static
buffer, so buflib_get_data() doesn't seem possible; but the compaction
callback can detect this and act accordingly. Since resample() doesn't
yield no CA is expected and compation should work without much hassle.
Alternatively small_resample_buf could be made a buflib allocation as
well, as it's unused if timestretch is enabled. Then resample()
could call buflib_get_data() unconditionally. However, for speed,
small_resample_buf is allocated in IRAM so it has a downside.
(buflib allocs from IRAM with a separate buflib_context perhaps?).
filetypes_strdup(): buffer_alloc() for each call. It allocates a string
buffer for the argument to hold a copy (i.e. like standard strdup)
It is called from filetypes_store_plugin() and read_config(),
each only called from filetypes_init(). It's probably called often,
for tiny sizes, which is not ideal for buflib.
The buffer is used for the in-memory representation of viewers.config,
setup at boot. the result of filetypes_strdup() is stored in
in filetypes array. No pointers are exposed to other modules,
except in openwith_get_name(), which is pretty safe as it's called
by the list drawing (if it was unsafe, it could be copied into the buffer
passed to openwith_get_name()).
The filetypes array could safe the ids instead of raw pointers,
the buffer is almost exclusively used in filetypes.c (see usage),
so buflib_get_data() calls should work just fine. But the filetypes
array can also walked through in the compaction callback. Compaction
should be no issue here, but the tiny allocations are suboptimal.
theme_init_buffer(): On native targets, buffer_alloc() the skin buffer with
a fixed size. The function is only called on boot in init(). It seems
to be replacable by a static allocation. On application targets malloc()
is used and thus isn't relevant for buflib.
usage: The skin engine has it's own memory management. One can allocate
from the skin buffer with skin_buffer_alloc() which returns a pointer
It's then up to the skin element what happens with that, it very probably
creates cross references (CR) into the buffer. The memory is also
exposed to other modules, such as in the backdrop case.
The skin buffer is reset upon theme (re)loading.
compaction: It looks like very extensive changes are needed in the skin
engine. Both CR and exposure to other modules are complicating
This needs more looking, but for now it seems the easiest to replace
buffer_alloc() with a static allocation.
apps/mpeg.c and apps/playback.c
audio_init(): buffer_alloc() for a cuesheet struct (~72K!!!), depending
on whether cuesheet support is enabled by the user. This allocation
holds the currently active cuesheet
The pointer to this cuesheet is saved to the current struct mp3entry,
and thus exported to other modules. struct cuesheet consists only of
several arrays and there's no CR involved. The actual cuesheet is mostly
passed to functions in cuesheet.c.
Additionally, the skin engine draws text from the arrays in the
cuesheet struct, however it seems to re-query the pointers before
compaction: The yield()s cuesheet.c is complicating things. Allocation
of the current cuesheet could be handled there entirely making it local
to cuesheet.c. Other modules could work with the handle id (also
in the mp3entry struct) and call buflib_get_data(). This way
the drawing by the skin engine wouldn't be problematic problematic.
However, SWCODEC works with other cuesheet structs (bufalloc()'d from
the audio buffer) and pointers (or rather buffering handles), so
those would need separate handling. At least the cuesheet struct is
moveable without futher means.
playlist_init(): 3 buffer_alloc() for the static current playlist struct.
playlist.indices and playlist.filenames are allocated according to the
max_playlist_size setting, playlist.buffer is allocated according to the
max_files_in_dir setting. playlist_init() is called only once during boot
in init().
indices seems to primarily save
the seek position into the playlist file the playlist is based off, or
the offset into playlist.buffer in case of an in-ram playlist.
buffer is apparently used to save filenames for in-ram playlist. It
is only accessed as start of an array, with playlist.buffer_end_pos
indicating the current position(as offset). Only playlist_save makes
a copy of playlist.buffer.
playlist.filenames saves pointers to dircache entries, which itself
are buflib affected, but that doesn't matter here.
indices and filenames are both exposed to other modules in
playlist_create_ex() (if index_buffer == NULL). However, this code
path is never executed, because the only caller calls it with
a non-NULL index_buffer-
If the unused code path in playlist_create_ex() is removed compation
should be relatively painless with a callback. It seems bad practise
to assign that buffers to a foreign playlist anyway.
scrobbler_init(): buffer_alloc() to alloc a cache of (string) entries,
depending on the enable scrobbling user setting.
usage is straight forward. It's only used within scrobbler.c, not exposed
and no CR. it's accessed as an array, no clever compaction for
short strings. Over time entries are added, when the buffer is full
it's flushed to disk.
Since it's accessed as an array, compaction should be painless. No
yield()s between access, so buflib_get_data() should work well.
tagcache_dumpload(): Exactly the same mechanism as in dircache_load().
allocate_tagcache(): buffer_alloc for the ramcache. The last size, plus
some spare for new entries (just like dircache w/ transparent build).
it's saved in the hdr variable which isn't modified. No buffer_alloc
if ramcache isn't available or deactivated.
The buffer has two parts, both filled in tagcache_load(). T
he first is the indices from the master index
it can be treated as an array of index_entry structs.
The other part is a collection of tagfile_entry structs including their
data (string value). For each tag a CR is saved in ramcache_header.tags,
from there you can walk through the indices with tagfile_entry.tag_length.
Pointers into the buffer are exposed via tagcache searches when
the result pointer is set to the appropriate tagfile_entry.tag_data
tagcache_fill_tags() also exposes lots of pointers.
Since tagcache loads from disk (the on-disk representation knowns no
pointer) it's generally well prepared for moving data. tagcache_dumpload()
also shows this and how to fix CR if the buffer position changed
(the same could be done in the callback).
The exposure of pointers is a bit tricky. For non-ramcache a static
buffer is used for the result. So it's perhaps best to use that
for ramcache too rather than returning the pointer directly (or
demand a caller supplied buffer). for tagcache_fill_tags(), the
mp3entry.id3v2buf member could be used.
tree_mem_init(): 2 buffer_alloc() for the entry buffers. The first creates
and array of struct entries, the second creates an string vector.
The buffers are actually used by filetree.c and tagtree.c. The former
builds a cache of the current directory, mainly for sorting purposes.
The latter does the same, however it does use the a struct tagtree_entry,
which is coincidentally of the same size (assuming padding), but
otherwise incomaptible. In both cases, the name member of the struct
points to the correspinding string in the second buffer (or sometimes
to static storage).
* tree.c: The strings in the name buffer are passed to a couple of
functions, like the list drawing routine, talk_spell(),
* filetree.c: the two buffers are filled ft_load (subject to file
filtering), then sorted according to the user setting. Other than that
it's passed to ft_play_playlist(), playlist_add() and str*cmp() a
few times. The entry structs are used in a couple more places, each
time used as an array.
* tagtree.c: the two buffers are filled in retrieve_entries() and
load_root(). Here's then name member points more often to static
storage. Again, the entries are passed to qsort() later on. It's
the buffers are also indirectly passed to tagtree functions.
At first sight compaction seems not so difficult. The entry buffer
is an array and only accessed as such, the second contains strings
only and is freely movable. However, the exposure to {file,tag}tree.c
complicats things. At least yield()s doesn't seem to be much of a problem,
and there aren't actually too many places querying the buffers, as it's
mostly looped over.
What could become difficult is the fact that the name member often points
to static storage, which must not be fixed up in a potential
compation callback, while the CRs into the name buffer need to.
general: The interaction between these files shocked me, and I started
refactoring in a separate branch. This work will hopefully make
the buffer handling be simpler and more buflib friendly.
tdspeed_init(): 4 buffer_alloc() for 2 overlay buffer and 2 output buffer
for the timestretch dsp effect, only if timestretch is enabled in
the settings.
The overlay buffer is some kind temp buffer to mix subsequent samples.
Even though it's base is assigned to tdspeed_state_s:ovl_buf, it's
used privately in tdspeed.c
The output buffer are only referenced in tdspeed_apply(), however it's
assigned to an outgoing pointer array which is used in dsp.c
Both buffers are only accessed as arrays
It's problematic that outbuf is exposed to dsp.c, because the caller
uses them and yield()s inbetween. Perhaps dsp.c should query
*reset_state(): 1 buffer_alloc on SWCODEC for the "thumbnail" buffer, only
if it hasn't been already allocated. Thus only if the caller is
Things are a lot different for HWCODEC. The entire audiobuf is used
directly for the thumbnails (music and voice can't be mixed on HWCODEC)
*load_voicefile(): Direct audiobuf allocation to load the voicefile
partially or in its entirety. If partially loaded, then memory is set
aside after the voice file header for a limited number of clips.
Both buffers are used mostly privately and accessed as array. The only
exposed data seems to be when the voice files are actually do be played,
then it's passed to the voice thread (or directly to HW on HWCODEC).
The direct audiobuf use is due to reloading voice files on language
change. Effectively the voice file buffer is re-allocated then.
A mechanism to steal the talk buffer exists for buffer_alloc()s
after the initial allocation.
NOTE for HWCODEC: Access to the MP3 hardware is exclusive, so only one
one of audio, voice or talk clips can play, this is why each of those
generally claim the entire audio buffer.
Compaction doesn't look very complicated as the buffers are private
and used as arrays. However, when actually playing the voice, special
care must be taken. Especially on HWCODEC, where a ISR calls a callback,
the data may not actually move (probably).
read_clause(): 2 buffer_alloc() for a search clauses' string. Only one or the other
happens, and the first one is fixed size.
get_format_str(): 1 buffer_alloc() for the format string (to be displayed
in the db browser)
add_format(): 2 buffer_alloc(), 1 for a display_format struct and 1 for
a tagcache_search_clause struct
get_condition(): 2 buffer_alloc(), bothfor tagcache_search_clause struct.
parse_search(): 1 buffer_alloc() for a struct menu_root
parse_line(): 3 buffer_alloc() for two struct menu_root and a struct
tagtree_init(): 1 buffer_alloc() for the uniquebuf
struct menu_root has a char array for which is used for the title
to display in lists (tagtree_get_title()). It also has a pointer array
holding pointers to struct menu_entrys. struct menu_entry has a
similar char array for the entry names. A pointer to that array is stored
into the struct tagentry (load_root()).
At the same time when a struct menu_entry is allocated, a struct
search_instruction is also allocated (so they're not allocated
The struct tagcache_search_clause, including their allocated str member,
are passed to the tagcache for searching and checking (tagchache_{add,check}_clause()).
They are added to the tagcache search in loops according to the
"dirlevel" in the tagcache browser so that search results are filtered.
There are potential yield()s when the tagcache search runs.
struct search_instruction is similar to struct tagcache_search_clause,
aiding tagcache search (e.g. it has the tags to filter). The name
member seems to be entirely unused and can probably removed, and nothing
seems to be be exported to other modules.
struct display_format saves information about displaying search results,
the name member of the struct seems only be used during parsing, after
that the group_id because format strings can share the identifier. Nothing
of it seems to be exposed to anything (NOTE: add_format() doesn't
check for TAGMENU_MAX_FMTS).
The uniqbuf is only allocated and directly passed to tagcache where
it's tied to a tagcache_search struct. Tagcache doesn't do a lot with it,
it just adds unique indexes to the list if there weren't some to
eleminate duplicate entries.
The title char arrays in the structs are possible a bit problematic,
because they're passed to the list. The pointer stored in tagentry
should possibly replaced by some other logic because the menu_entrys
are allocated separately from tagentry. the struct search_instruction and
struct display_format seem painless to convert. Most problematic seems
tagcache_clause because compaction potentially happens during a
tagcache_search in which it will be tricky to move the structs.
OTOH tagcache shouldn't need to know whether the structs are buflib-backed
or not so it needs to be resolved within tagtree.c
The MM in tagtree.c is overly complicated with several allocations
happening, it needs more looking into later
audiobuf accessors:
plugin.c, mpeg.c, talk.c, playlist.c, tagcache.c, main_menu.c, playback.c,
mp3data.c, dircache.c, buffer.c, rolo.c
playlist.c: HWCODEC only, grab talk buffer + audio buffer temporarilly
SWCODEC calls audio_get_buffer()
- replace with something similar to clipboard mangement,
on-stack buffer should work
plugin.c: Same as in playlist.c (for both HWCODEC and SWCODEC), expcet
that it grabs temporarilly for plugins
dircache.c: Uses it in on-boot initialization as "get all, I'll give you
something back later"
main_menu.c:read-only, for displaying the current audio buffer size (non-critical)
playback.c: implements audio_get_buffer(); optionally turning down voice
and grabbing its buffer also. returns audiobuf to the caller
tagcache.c: see dircache, for a temp buffer during committment of