Test suite wishlist
This page is meant as a TODO list for new contributors interested in the test suite, and for ourselves. Feel free to tackle any of these items, to add your own, or ask for help/guidance.
Dogtail tests ("tests/dogtail_scripts/")
- Our UI tests cover a decent amount of functionality, but they can always be improved or extended!
- Many of our tests are a bit naïve. They need to be a lot more sadistic. Try with hundreds of files, with weird filename and paths encodings such as "/home/foo/bar baz/いろいろな紙袋とねこ。-Various paper bags and Maru.mp4", and try the various operations repeatedly. For example, don't try splitting a clip only a handful of times: do it a hundred times (with randomized positions/times in a "for" loop, etc.).
- Fix the makefile to also run the dogtail tests (as we need to run them manually with python right now)
- Eventually, if we replace our menubar by an appmenu, we’ll need a way for dogtail to access those items easily. Either a) we need to be able to sniff gnome-shell easily, or b) we need to provide keyboard shortcuts for every action in the appmenu for dogtail to trigger (might be easier/faster). See also bug 683748.
- Generally speaking, the timeline tests need a good cleanup. It would be really nice to find a way to make our various timeline objects (clutter actors) accessible with Clutter's accessibility implementation - for instance, Clutter has a clutter_actor_get_accessible() method that we could use. We need our timeline to be accessible for people with disabilities and for our Dogtail test suite to run more easily and reliably.
- Tests for still images: insert a video clip, a png image, the video clip again, a jpeg image, the clip again. Then, move all of their trimming handles around, also seek in various places at the same time...
- Media library:
- Make it test both the treeview in addition to the iconview... not super easy to know which one is currently showing. Maybe the only way is to have two test cases, and toggle the view with the menu items at the start
- Tests will all fail if the user’s preferences is set to have the media library in listview mode instead of icon view, because nothing is selected by default so you cannot use “Insert at end of timeline”.
- Test drag and drop import from other apps (such as Nautilus)? that's quite complex
- Fully test the Project Settings presets functionality
- Fully test the Rendering presets functionality
- Test the "relationship" between rendering settings and project settings (that stuff doesn't propagate back up to the project settings
- Loading projects with missing clips
- The relationship between "Project settings" and the "Rendering" dialog (note: when implementing this, it would be good to add the following rules/principles as comments in the test script's code):
- In the rendering dialog, some things are meant to be remembered: the resolution "percentage" scale, the chosen container/codecs, the settings of the chosen codecs, the render output folder
- ...and some things are meant not to be remembered: project settings "influence" things such as framerate, number of audio channels and their sampling rate, etc... but these settings are meant to be temporary overrides for a given render. Once you're done with the render it will reset to the project settings defaults.
- Resolution in project settings influences the resolution in the rendering dialog, which only has a "percentage" scale and a button to change the project settings
- Preferences: the test should properly revert the preferences to the defaults as a teardown at the end of all the tests, to prevent messing with the user config outside of the test suite too much
- Add your ideas here
- Removing clips from a project
- Changing project settings
- Timeline operations:
- Splitting clips on the timeline
- Trimming, ripple/roll editing
- Changing the transition type and parameters
- Move a clip vertically. Then, undo/redo 10 times in a row. No error should occur.
- Removing a layer
- Import two clips to the media library, then insert them onto the timeline. Undo 2 times (remove the 2 clips from the timeline) and undo another 2 times (remove the 2 clips from the media library). Then redo 4 times.
- Changes applied to effects/clip properties
- Add your ideas here
Normal tests ("tests/")
TODO: we should make a list of items that need to be added to this wishlist.