Design notes

The layout of staffs/lyrics/voices/page breaks between movements etc is not so easy to visualize and control in Denemo. We have a command to print the score and one to print a part and some more specialized ones to print reduced scores etc. But hidden away (in the LilyPond view) is all the mechanism needed to print any sort of score from the music data that Denemo holds. (This is the custom score block facility, which requires LilyPond knowledge to use).What we could do is provide a Score Layout View. It would hold the blocks of music data (the staffs, voices, lyric blocks etc) displayed as widgets. The user could then drag these around as they wished, omitting unwanted parts, inserting page breaks between movements, positioning Lyrics above the staff, enclosing groups of staffs in the staff groupings (Choir, Piano, Grand etc) and name the particular score layout for future use. Denemo just has to create the LilyPond custom score block corresponding to the layout the user has created – all the rest is already done. Well, not quite: to store and reload from disk we will not want to parse the text of the custom scoreblock. Instead we will need to save the (hierarchy) of widgets.In this case commands like PrintPart would just create a layout view containing only that part, and then use that to print.The motivation for this idea was a request for a SATB choir with the parts on two staffs, the four sets of lyrics distributed above and below the two staffs. Denemo does not know how to handle lyrics on voices on that share a staff – I am not sure what it would do, but it couldn’t be pleasant!

The score layout view would provide a more obvious place to edit the score titles etc and to set the indent and other aspects of the score, since the blocks would be identified by these fields (unless they were empty, in which the numbering would be used – movement 1, voice 2 etc). The user seeing these names would naturally think to edit them directly there – for example, editing all the titles of the movements in this one window, rather than navigating through the movements and editing there.

We have had point and click for some time – you click with the mouse on an object in the final typeset score (in the print preview window) and the Denemo cursor is moved to that note in that staff in that movement.

What this technology will also allow is placing references to source manuscripts, facsimiles at points in the Denemo score. At present we can store snapshots of a original source manuscript of the music with the measures. This requires quite a bit of storage and is done for the whole of the piece.
Instead, we will be able to place “links” to the manuscript which when clicked on will open the pdf at the given location (highlighted) so that the editorial process can be easily reviewed. The links can be inserted by the inverse process, clicking on the pdf at the point we are about to transcribe will insert the link, as well as giving a marker in the source pdf to show where we are.
The source manuscripts/facsimiles would be displayed in a different window from the print preview window, of course, but most of the programming is the same – just more evince widgets and some filechooser to load them up.

When transcribing a piece from audio (e.g. an ogg or mp3 file) it would be nice to be able to do the work with the audio linked to the score. So, for example, one could write a chordal accompaniment and playback the original audio and the denemo_wordpress-generated MIDI. The key to doing this is synchronizing Denemo’s playback with the audio (which is assumed to be real (human) music whose tempo will vary slightly.

This synchronization is possible with the control Denemo has over the live playback of a score (see e.g. the Conductor feature in the playback controls). What else is needed is a means of reading an audio source and playing it back with a Denemo score. In principle the libaubio could provide beat information, but I suspect this is a research topic. What would be very practical would be for the user to input the beats by pressing the space bar while the audio is playing, which would give Denemo a basic rhythm track which it would use for subsequent playback.

The simplest use would then be to write the proposed transcription in Denemo and when playing back any discrepancies would be easily audible. With this in place it should be also possible to play in via MIDI controller during playback – so you could play along with the original audio with a MIDI recording being made of your contribution. Then a scheme script (like the current Record and Convert script could be used to covert to notation.

To implement this libsndfile could be used to open the audio original. Also of interest would be librubberband to change the tempo of the incoming audio.

This is now bare-bones implemented:
With the File->Open->Open Source Audio command you can select a file (must match your frame rate, and have two channels) to mix with the usual Denemo synthesizer output. The “mixing” is done in the crudest possible manner, taking one stereo channel from each.
When you have source audio loaded, additional widgets appear in the playback controls to set the volume relative to the synth output, and to set the lead-in – delay or trim before the audio starts.
Under the Playback->Audio menu are commands to perform bar-by-bar synchronization. You press the space bar once for each bar in the audio, and Denemo creates a timing track – a staff with a whole measure rest and a tempo setting for each measure.
On my machine the accuracy of this synchronization is only a few percent, but my machine is very old, please let me know how this works on your machine. It may be that the synchronization script should listen for every second bar or so.

The original motivation for this feature is creating a transcription from an audio source. Once synchronized, when you press play the score and the audio playback in synch. You can create a loop, editing and listening while the music plays back.

Other possibilities

• I imagine that it would be possible to create extra written parts to an audio track using this feature.
• It would be possible to output audio in the same way that we currently output MIDI notes (e.g. on entering a note we output the pitch via MIDI). So, for example, we could output spoken/sound-effect feedback on change of prevailing duration, start of triplet etc. At the moment, we are limited to GM Midi instruments, as used for example for the rhythm feedback.

We need to protect our users against trying to print or play files with incomplete “spanners”. “Spanners” are things that span notes (such as start repeat section end repeat section, start tuplet end tuplet, start cresc end cresc…). We might extend the concept to things that span ‘vertically’ such as start piano staff end piano staff.The sort of operations a user can do to damage a spanner is to delete the start or end object (possibly a note with directive attached), to delete an object enclosed in a span, to cut a range of objects including one end of a span. To paste back such a cut, possibly now enclosing more objects, possibly now with the end span pasted before the start span.
The simplest thing would be to run a check before typesetting or midi-creation. Such a check routine would need updating every time a new sort of spanner was introduced by scripters.Better would be some spanner support. Simplest would be some convention for tags that indicate if the directive is the start or end of a spanner. (And then more elaborate schemes such as might be used for repeat, first time, second time constructions). Again these would be used by a check routine, but now a new sort of spanner could automatically be checked by creating its directives following the convention.
Better still – but needing some real design work – would be a flag to mark DenemoObjects as being involved in a spanner. These would trigger a callback to when something happened to the object. The callback could be the original command used to create the spanner with a parameter so say what was being done. So the one script handles both creation and what to do on deletion. Handling what to do on object creation (extra objects within the span) would also be possible – but it could become difficult to avoid a performance hit without building elaborate indexes (where spans start and end).
With some support it would be possible for the callback to indicate if it was d-Cut or d-DeleteObject that was causing the deletion. How that information could be used is moot.

When working from a source score in facsimile making a transcription a good way to associate the source score more directly with the transcription would be as follows:
Each measure would have attached to it the measure of music it transcribes. It would make the transcription process easier, as the source material would be presented next to your transcription.
This would have another merit – a user could consult the original material if they doubted your editorial decision.So in this scheme you would first of all go through the original print dragging a box over each measure – each time you clicked a measure would be created in Denemo with the image of the measure the user has outlined attached.
Then, when playing a measure in, the original would be visible immediately below the current measure.There is a parallel here with the audio score mixer above. In each case I envisage the user dividing the original up into measures and then adding stuff in Denemo that corresponds to the original, be it an accompaniment or a transcription.
Here are some snippets of code for the gtk-screenshot program that would be needed for this:

screenshot_select_area (int *px,
int *py,
int *pwidth,
int *pheight);

screenshot = screenshot_get_pixbuf (window, rectangle, include_pointer, FALSE);

job->window = find_current_window (&window_title);

find_current_window (char **window_title)
window = gdk_get_default_root_window ();
tmp = screenshot_get_window_title (window);

gdk_pixbuf_save_to_buffer ()

We are currently using xml to save denemo scores, this would be a verbose way of storing the pixbufs, even when converted to .png using the above call.

Now that we are getting more menu items it is becoming more useful for a user to be able to collect together their own frequently used commands in a custom palette(s) or button/toolbars.The position and ordering, commands stored and name of palette would all be stored in the user’s command set file ~/.denemo/actions/Default.cmdset.
Ideally the user would be able to drag and drop items to any of his custom palettes, and re-arrange them once there, but as a first implementation they could be placed in palettes by a menu command (on the right click menu).The place where the code to handle this would need to go is marked (in the src/lilydirectives.c source file) as//FIXME at this point you could allow the user to specify a custom button box for his directive – some property of the directive saying which button box it should be in. We could even allow the directive to create a toolitem of a toolbar or menuitem on a menu bar???