Journal

/A/ /M/A/G/N/I/F/I/C/E/N/T/ /P/L/A/C/E/ 

July 2012

23:38 Wednesday 18 July 2012

Petri-Foo development

Since my last post here, I eventually managed to stop feeling quite so sorry for myself and pick up the programming bug again. Work in the factory has been tough now that I feel like this is it, my life now, factory worker. I can't see beyond it. My motivation there is paper thin. I've started taking a lot of notice of how my muscles feel and have used that to pace myself. The slow pace is anti-training, breaking the habit of working as fast as possible out of fear of reprisal. Minimum wage minimum effort mantra.

It's not always the case. I don't always feel so negative at work. Just half the time. I'm going through a stage at work where I no longer accept that I only get one break a day. It gets to the point where enough is enough repetition and for my own sanity I walk away from the machine for a few minutes to get a coffee or something.

At home I've been working on Petri-Foo. I am currently implementing support for Non Session Manager which is proving quite a substantial chunk of development time for me, though it should be mentioned the fault for that cannot be totally directed at NSM's behavioural guidelines more that Petri-Foo was not as ready to be accommodating as it could have been.

================================
***** petri-foo save types *****
================================


this document's aim is to bring to light the complications involved in
implementing a new save type for petri-foo which arises from dealing
with session managers having rules and regulations for how to deal
with state saving and in particular, files external to the session.

the purpose of this document is primarily to layout and clarify the
different situations which arise from this new save method and identify
problems involved in dealing with it.

while it is true the implementation being worked on and discussed here
arose due to wishing to support session management, the method does not
loose any practical relevance without session management.


----------------------
***** save types *****
----------------------

1) quick-save
    this creates only a .petri-foo file, samples in the file are referred
    to by their full path. no complications. this is the current method.

2) full-save
    * in a user-chosen directory (bank_dir)
    * create sub-dir named samples/
    * (for each sample) create a sub-dir of samples/ named using
      SHA1 hash of the full path to the dir containing the actual sample.
    * symlink to the sample within hash dir.
    * convert absolute sample paths to relative paths referring to symlinks
    * save bank.petri-foo in bank_dir


------------------------
***** sample paths *****
------------------------

the first encountered problem is how to deal with sample paths for
symlinks. at the time of writing this document, the problem has been
solved by the use of SHA1 hashes.

here is an example of how paths can become a problem for symlinking:

/Storage/Samples/ElectroKit/snare.wav
/Storage/Samples/RockKit/snare.wav

there will be a symlink name clash here as both symlinks would be
named snare.wav because the file names do not differ other than by
their path on the file system.

i first thought to recreate the path of the file or to perform a text
translation on the path so it could become part of the symlink name but
both of these lacked appeal. then i hit upon the hash idea (after reading
about using SHA1 for identifying content in a sample).

other existing implementations use a much simpler method of adding a 
numerical suffix to the symlink, but the implementation i investigated
did not check to see if the sample to be linked to had a symlink pointing
to it already. i felt that to be unsatisfactory and decided checking for
a single directory (ie the hash) to be better than checking every symlink
in a directory. additionally, i wanted some method to preserve path
information. with the hash method i create a text file within the hash
dir which contains the a string showing the originating path - i lean
toward believing this to being more useful than not useful.


---------------------
***** full-save *****
---------------------

the easiest way to begin to see the complications is to break
the process down into stages. this nescesitates including the
open process.

1)  nothing is saved - when asked to save, the save process runs through the
    steps as described.

2)  a full-save bank is opened - when pretending ignorance to any forseable
    complications we can say the open process remains unchanged.


now may be a good time to mention the purpose of this new save-type is to
allow archival of banks. use of symlinks (as advocated by David Robillard
and others on LAD and elsewhere) pointing to samples allows archiving by
tar -h. this means eventually, an unpacked full-save archive will contain
copies of the samples rather than symlinks to them (thus allowing the bank
to work on some other machine etc).

but a condition has arisen:

* samples existing anywhere within bank_dir are not external so must
  be ignored by the path-hash and symlink process but they do still
  require to be referred to by a path relative to bank_dir so as to make
  the bank movable.

i suspect now you're starting to see complications arise like the undead on
halloween...


--------------------
***** PROBLEMS *****
--------------------

(1)

the first implementation (as of time of writing) has caused state saving
functionality to break out of libpetrifui and contaminate libpetrifoo.
this is to be avoided.

the contamination came about after it was seen that the original
dish_file_write function could still remain usable for full-save
functionality if the process was dual staged. firstly,
dish_file_dir_write would create directories and symlinks and
modify the sample paths in memory.

unfortunately, the Sample structure is not accessible outside of the
private libpetrifoo implementation which meant creating an accessor
function to modify the sample path ready for dish_file_write.

(2)

the second problem comes after a user has loaded a full-save bank, makes
modifications, and activates save - the save process needs to be aware
which save type it is to perform.

perhaps the save type could be gleaned from looking to see if the sample
paths (as stored in memory) were relative to the directory containing the
.petri-foo file?

alternatively, it might simply be better to specify the save type in the
save file itself when saving it!

this information can become a property of the root node like so:

<Petri-Foo-Dish save_type="full">

 - or -

<Petri-Foo-Dish save_type="quick">

the latter type is the fallback type for those files where the save_type
property is unspecified.

(3)

full-save-as after loading a full-save bank. relative paths to symlinks
need identifying and dereferencing to the original file to avoid
symlinking to symlinks.


---------------------
***** SOLUTION? *****
---------------------

When loading a full-save bank, the relative paths must be converted to full
absolute paths (there's no messing about with "working" directories here).
there is not (as of time of writing) any conversion from symlink path to
actual sample file path.

however, by converting from symlink to actual, the full-save-as problem
is solved:

* conversion from symlink happens on load
* conversion to symlink happens on save

this removes the contamination of libpetrifoo by state-save code, but
requires dish_file_read and dish_file_write to know the save_type.

* conversion from relative path to full happens on load
* conversion to relative from full path happens on save

again, this also requires knowledge of the save type, but additionally,
knowledge of the bank_dir. a static variable in dish_file.c last_bank_dir
can be used to indicate the save type as full when it is set, or quick
when it is not.


----------------------
***** QUESTIONS? *****
----------------------

* should full-save-as allow overwrite?
    * should anything be deleted?
    * can it overwrite the bank it opened?

* should last_bank_dir ever be ignored or reset?

    last_bank_dir will be set under the following conditions:
        * when dish_file_write_full is called
        * when dish_file_read is called and opens a full-save bank
        * when started under NSM session management

    last_bank_dir should be cleared under the following conditions:
        * when new bank is selected and not under session management








  

Information

"Journal"

A page detailing new stuff and other random noise.

The journal is a general place for writing about what I am doing, or for making more official announcements concerning the things I do. It's also a place where I can write freely about my ideas, or just play with words and language.

DISCLAIMER: The opinions and attitudes of James W. Morris as expressed here in the past may or may not accurately reflect the opinions and attitudes of James W. Morris at present, moreover, they may never have.

this page last updated:29th April 2013 jwm-art.net (C) 2003 - 2017 James W. Morris

script time:0.0597