path: root/
blob: 288f3b9754f9667a8f97a2aa29b8d51f72848cb4 (plain)

Using digital media masters

Master files - either original sources or as close as possible - are precious and should be carefully tracked.

Digital media files - e.g. photos, videos, music - are often large and binary encoded, which poses special challenges for their tracking.


Digital media masters are tracked with git and git-annex.

Git is a version control system - a repository for historic versions of file contents of a folder. Git-annex is an extension to git, better handling large files.


You are recommended to use path suffix "annex" for media master projects. Examples:

  • ~/public_annex/home-videos
  • ~/private_annex/friends-snoring
  • ~/shared_annex_family/xmas_photos


To turn a folder into a git repository (see also alternative of (cloning)[#Clone] an existing project), go into the folder, and initialize its git and git-annex databases:

git init
git annex init --version=7

To use git-annex only for large files (git for smaller ones), add e.g. the following to file .gitattributes:

* annex.largefiles=((largerthan=100kb)and(not(mimetype=text/*)))
*.svg annex.largefiles=nothing

Finally (save)[#Save] all content:

git annex add .
git commit -m "Initial commit"

(final git commit is implied by git annex sync or git annex move)


To check status of metadata, use git:

git status

To check status of file content storage, use git-annex:

git annex info


To "take a snapshot" of one of more files for git-annex archival, first mark which files are involved and then archive their (changes to) content:

git annex add foo bar
git annex add baz
git commit -m "Update foo bar, and add baz."

(final git commit is implied by git annex sync or git annex move)


To collaborate on a shared git repository, first create a local clone from the shared location, and tell git-annex to use it:

git clone git://[[!template id=githost]]/example
git annex init --version=7

Then from time to time syncronize, ensuring that all content exists both locally and remotely:

git annex sync --content --all

Alternatively (e.g. on slow/expensive network), syncronize only metadata and only with nearest clones:

git annex sync

Alternatively (e.g. on small host), push the content to only be remote without keeping a local copy:

git annex move .


To publish a git repository initially created locally, first create a new empty git and git annex publicly, then tell your local git where its new origin will be, and finally push your local git and git annex into its new public location:

ssh [[!template id=githost]] git init --bare --shared /srv/git/[[!template id=githost]]/example.git
ssh [[!template id=githost]] GIT_DIR=/srv/git/[[!template id=githost]]/example.git git annex init
git remote add origin [[!template id=githost]]:/srv/git/[[!template id=githost]]/example.git
git push --set-upstream origin master
git annex sync --content --all


While main purpose of git-annex is to store large media elsewhere, you obviously want to sometime work with the content locally.

To fetch content of current folder and all subfolders:

git annex get .

To afterwards relieve local storage of git-annex tracked content in current folder and all subfolders:

git annex move .

To fetch or relieve a single file or another directory, replace the dot with the path to the file or directory.


Splitting media files is format-specific.

Many audio formats can be split with the package shntool and appropriate helper tools, e.g. like this for a FLAC file with CUE file for splicing hints:

shnsplit -o flac -f CUE_FILE.cue FLAC_FILE.flac

For many other formats, including video content, splitting can be done using the copy parameter to FFMpeg, e.g. like this:

ffmpeg -i INFILE -ss 15 -t 60 -acodec copy -vcodec copy OUTFILE


  • [media-master][Source of this document]

[media-master]: "Digital media master material"