2008 UI Design

From Pitivi wiki
(Redirected from UI Design)
Jump to: navigation, search



To Be Done


GIMP file(s) used for creating mockups


Design Principles

  1. Use direct manipulation for most common operations.
  2. Use noun-verb pattern for most other operations (see below)
  3. Minimize the number of modes. Modes are evil.
  4. Use quasi-modes instead of modes
  5. Provide feedback. Change cursor styles, provide real time updates.

Noun-Verb Interaction

Select what you want, then issue a command to manipulate it.

Representative Tasks

Frequent Tasks

Occasional Tasks

Special Tasks



There are several primary components:

These components all share a single window by default. The Viewer, Timeline, Clip Library, and Clip Editor, and Effect Library can be detached from the main window by clicking a special "detach" button. These windows appear as normal top-level windows. When the user closes one of these windows, the component returns to its default location. In addition, the clip editor and viwer can be "expanded" so that they completely fill their parent windows.




When the mouse moves over a selectable item, it becomes highlighted to indicate its focus.


Single-clicking on items sets the selection to that item. Selection is indicated by more prominent highlighting.

Whatever the method used, selection is an undoable action.

Select4.png Select5.png

Click and drag on blank canvas activates selection marquee. While dragging the selection marquee, objects that will be selected when the mouse-button is released indicate focus. It is clear which objects touched by the marquee will be selected.

Select basic.png

Some tool-bar commands to modify selection. These are track-wise operations. The operation will be performed for each track which contains a selected clip. These operations to not apply to selections of key-frame points within clips, but they do apply to timeline and track-level markers.

Select before.png

"Select before" -- select everything between the start of the project and the current selection

Select after.png

"Select After" -- select everything between the end of the project and the current selection

Select above.png

"Select Above" -- select everything between the top-most layer and the current selection

File:Select below.png

"Select Below"

Select layer.png

"Select Entire Layer" -- selects everything else in the layer(s) of the currently selected source(s)

Select track.png

"Select Entire Track" -- selects everything else in the track(s) of the currently selected source(s)

Select none.png

"Select None" -- remove all items from the current selection

Select all.png

"Select All" -- select the entire project


Used for playing back video from a variety of sources, primarily the timeline. Provides basic playback controls, as well as single-frame forward / rewind.

Functions of the Viewer

Automatic Preview

During certain edit operations, the it would be useful if the viewer could show visual feedback of the edit in progress. After operation ends, viewer returns to previous state.

Continuous Loop

The user can lock the viewer into a continuous loop over a portion of the timeline. In this case, all other seeking behavior is disabled. The user can make continuous adjustments while the loop plays.


Timeline detail.png

The primary component of the UI. This is where the user directly manipulates sources and effects which will appear in the final output. It is a time-proportional representation of the edit decision list. Clips appear as horizontal rectangles, their with proportional to their duration, and their position along the x axis proportional to the time at which they start. The metaphor is of virtual strips of film. The user can "cut" (meaning split), move, resize, and group (splice?) these strips.

Seeking and Navigation Controls


Multi track editing.png

The timeline is subdivided into tracks. Tracks represent separate output channels with a single media type. Within a track, all objects have the same media type, and there is at least one track in the timeline for every type of media that the timeline contains. The user does not add or remove tracks directly: the user adds objects to the timeline, and tracks are created as appropriate. Most projects will only have one audio and one video track.

Tracks can be expanded or contracted. Expanded tracks stack their clips vertically, according to the clips' layer position (priority). When a track is collapsed, all sources appear at the same vertical position, as if contained within a single layer. This single layer can then be further collapsed.


Tracks are themselves subdivided into layers. Layers are priority levels within a track. For audio, all layers within a track are mixed together into a single stream. For video, all sources within a track are composited together in a single stream. The layer position determines the order in which videos are composited, with the visually topmost layer appearing as the top-most source in the stream.

File:Layer addition.png The user can add layers to a given track by dragging the track's separator bar downward. Similarly, the user can remove tracks by dragging the layer's separator bar upward; however, removal of the bottom layer will only be permitted if the layer is empty. Objects within layers can be stacked arbitrarily. This is particularly useful for effects, which operate lower-priority objects within the timeline.

Managing Vertical Complexity

Expanded Layer

Ordinarily, layers take up a fair amount of space. This is to make room for thumbnails, waveforms, and keyframes.

Collapsed Layer

Layers and tracks can be contracted to save space. The user can contract a layer by clicking their expander widgets on the far left side of the timeline.

File:Contracted track.png

This can also be done for entire tracks

Sources and Effects

Timelineobject detail.png

Sources and effects (within a track) are content streams of a single media type. Sources are clips which provide data. Effects are filters which consume lower priority clips as input and produce filtered output.

Both Source and effect objects have properties. All properties can be manipulated via the property browser, but some of properties, like audio volume or video alpha, will be so commonly used that they are embedded directly onto the widget. These "embedded" interpolation curves are manipulated in exactly the same way as interpolation curves in the property editor (see the Default Property Editor section) for more info.

Moving Timeline Objects

Objects in the timeline can move in both horizontal and vertical axis. The semantics, however, change depending on the type of object. For all objects, the horizontal (x) axis is interpreted as the time axis. For sources an effects, the vertical (y) axis the source's layer position within a track (tracks are shown visually stacked, but moving a source between tracks is accomplished through a different mechanism).

Moving Horizontally

Edge / Frame Snapping

All objects have edge snapping enabled during horizontal motion. At this point, we believe this is the most common use case. This edge snapping effect is intended to be subtle, with a deaband of only a few pixels.

Temporary Deactivation

File:Deactivated objects

Objects in the timeline can be temporarily deactivated. The deactivate command is in found in the timeline toolbar, and will deactivate whatever objects are in the current selection. The reactivate command undoes this operation.

An entire track can also be suppressed. To do this, click the disable toggle near the track's name on the left side of the timeline.

Adding Objects

Adding a Clip

File:Add clip1.png The user chooses the clip from the clip browser by clicking and dragging.

File:Add clip2.png When the object enters the timeline, the timeline responds by showing how the timeline will change. In this case the clip has both audio and video streams, so objects appear in both audio and video tracks.

File:Add clip3.png The user can move objects to desired layers and time offsets.

File:Add clip4.png By holding the appropriate modifier key, the user can push existing objects out of the way...

File:Add clip5.png ...or add a source into a new layer.

Adding an Effect

Effects can be dropped into the timeline in almost exactly the same way as clips. The main difference is that effects come from the effects library.

File:Add effect1.png User selects the effects library from the tab.

File:Add effect2.png When adding effects into a new layer, the layer is initially collapsed.

Fine Tuning: Trimming Objects

Trimming a clip is always possible by clicking/dragging on source trimming handles. By default, the in or out point of a clip should be edge-snapped (so that it is easy to put the clip back the way it was). The UI should constrain the the settingof in/out point so that sources can't be stretched beyond maximum native duration. clicking and dragging a trimming handle should not change the current selection


First, the user moves the mouse over the desired clip's trimming handle


The cursor changes to a left- or right-edge trimming cursor.


Click-and drag sets the in or out point of the clip as appropriate.

Roll Edits

A variant of trimming, which works when two clips are adjacent in the same layer. Sets the in-point of the left clip an the out-point of the right clip, keeping the total duration of both sources the constant. Roll edits are activated by holding the appropriate modifier key while dragging a trimming handle. Note that this is only expected to work when it is possible to set the in/out points of both sources to the same point in time.


First the user places the mouse over the appropriate trimming handle.


The user holds down the appropriate modifier key. Cursor changes from trimming to roll-edit cursor.


When the user drags the mouse, the edit points are set as appropriate.


However, if the user releases the roll-edit modifier key, the edit reverts to the default trimming operation.

Ripple Edits

This is another variant on basic trimming. The source who's trimming handle is being manipulated is trimmed as usual, however the adjacent source(s) are rightward in the appropriate direction, so that the trimming does not create a gap between the sources. This shifting carries down the entire length of the track, keeping sources in the same relative position.


User places cursor over the desired source's trimming handle.


User holds the appropriate modifier key. Cursor changes to ripple cursor


User drags the the mouse. Adjacent sources are shifted.


The user can also hold an additional modifier key to make the ripple edit work across multiple layers.

If the user releases the ripple-edit modifier key, the edit reverts to the default trimming operation.

Time Stretch

Another variant on basic trimming. The source's in/out points are not set as normal, but rather the source keeps the same in/out points and the source is sped up or slowed down to accommodate the new duration. Timestretch only applies to sources of finite length, such as files.


There are two methods of combining timeline objects together: linking and grouping. Linking allows the user to keep distinct timeline objects synchronized. Moving one object causes all of its linked "sibling" objects to move. The relative offset between siblings is preserved.

Link1.png Link2.png

Some clips will be linked by default (for example, audio and video from the same file).

Link1.png Link3.png

But the user can link arbitrary objects together as well.

To link objects:

To unlink objects:


Grouping is similar to linking in that multiple objects are combined, but different in that the resulting group is treated as a single object. The user can make multiple "clones" of a single "master", and changes to the master will ripple out to each of the clones. Unlike linking, grouping creates a new "clip" in the Clip Library. Effects applied to the group apply to the output of the group as a whole, rather than the topmost object in the group.


To group objects:

To ungroup objects:

TODO: how will we edit the groups? Two approaches: recursive editing, or "expanding" in place. What are the pros and cons of each? Other issues: full, partial, or no synchronization of clones.

Clip Library (formerly Source Browser)

File:Clip library.png

Contains a list of all the clips in the project. The user can drag external files onto this component to add them to the project (an import tool bar command also works. The user adds clips to the Timeline by dragging them them from the Clip Library and dropping them onto the Timeline.

The Clip Library also provides commands to manipulate the clips in the project:

Property Editor


Sharing the same tab view as the source browser is the property editor. While the timeline is meant to provide a film-strip metaphor, the property interface allows the user to change the more abstract properties of the currently selected timeline object(s) (for example, audio balance, or image color correction). The type of controls presented are determined by the current selection:

Default Property Editor

File:Default property editor.png


Features The default editor lets you set all of the otherwise hidden properties of an object. It's will be most usable when the mapping between an object's properties and their effect on the output is straightforward. For example, audio volume, our video alpha. The current implementation of the videobox element is a good example of what won't work well with this module (a more dedicated UI focused on cropping/panning would needed).

A few object properties will be static (i.e. they are time invariant). These will be displayed as standard GTK+ Widgets. Other properties are "controllable" (i.e. time-varying). The user can directly manipulate the interpolation curves for these properties through curve control points objects, which we refer to as "key-frames." A partial prototype of this design is available.

File:Properties curve.png

The user can add key-frames in two different ways ways:

The user can delete key-frame points in two different ways:

GStreamer supports different interpolation modes, but only for the entire curve. Changing the interpolation mode for a single point isn't possible (we'll have to write our own interpolation code). On the other hand, it makes sense to treat interpolation mode as a per-point option. For now setting the interpolation mode on a point will simply set the interpolation mode on its parent curve.

Image Property Editor

The user can add still images to the timeline. By default the image is letterboxed to the current project resolution, but these defaults can be changed to suit the users's needs.

File:Image source1.png First the user selects the image in the timeline

File:Image source2.png The image property editor appears in the property browser.

File:Imgage source3.png The user can crop the image to an arbitrary region.

File:Image source4.png

File:Image source5.png The user can scale the image as appropriate.

File:Image source6.png The user can also set the orientation of the image.

Animation Property Editor


Specify sets of pictures which will be displayed in sequence:

Other Ideas:

Motion Transform Editor


For the "motion transform" effect object (yet to be implemented)

Needs to support the following properties over time

Also need to be able to set the color and/or alpha of the "background".

Ideas...parametric curves on a 2-d plane? Looping previews?

Advanced Compositing Editor


Chroma key

Blue/Green/Red screen

Should thresholds and key-color be time varying? Definitely need at least a local preview. Using the viewer would be better.

Title Card Editor

File:Title card editor The title card editor might look something like this.

Effect Library

The effect library lists all of the available effects, whether delivered through plug-ins or internal to PiTiVi. The user can drag-and-drop effects into the timeline in the same way they can with clips.


Main Toolbar

Provides application-level commands:

Timeline Toolbar

Flush with the edge of the screen in full-screen mode, for easy access to commands.

Finally, there is one modal tool -- the split tool -- which splits a clip or effect object into two segments.

Zooming Controls:

Provides a list of commands which operate on the current selection.

In addition, other commands will appear or become sensitive depending on context - i.e. the current selection.

When one or more curves is in the selection:

Zooming and Scrolling

Preferences Dialog

This is a list of every imaginable setting that could possibly go in there. It is intended to be reviewed mercilessly, so that we decide to accept/deny their inclusion.

Personal tools