>Unfortunately, I have found
that user bits are hardly ever set correctly. >They are also
not well integrated into editing equipment.
User bits, that is 'date' and 'roll number' are the only way to
keep same TC numbers apart. You'll always have TC numbers showing
up more than once.
What are you suggesting to do?
Sound mixer, etc
Karl Lohninger wrote:
>User bits, that is 'date' and
'roll number' are the only way to keep same >TC numbers apart.
You'll always have TC numbers showing up more >than once.
>What are you suggesting to do?
I don't know what Noel is proposing, but what's done now is to simply
name the directory that the files are in, much as you name a roll
number on tape. The names can be based on whatever you want, but
the logical way to do it is to name the master directory for the
"roll number" and/or shoot date, and the "subdirectories"
for scene, and then take.
For final conforming, a combination of the timecode based EDL and
the "negative cut list" (which contains roll number, scene,
and take) is used to uniquely identify all frames. There is no actual
standard for this that I know of'; all current DI vendors have their
own ways of combining this information for the final assembly. All
file based systems have a hierarchical directory structure - they
To me, the complicated thing to do is to stuff all this metadata
into each and every file during live recording (i.e., production)
and make it all work in a way that conforms to some sort of standard
and is reliable. That, to me, seems quite a ways off, both in terms
of reliability and simply deciding on a standard, regardless of
what the AAF people seem to say.
IATSE Local 600
> I don't know what Noel is
You and me both...
For years I have been struggling with "timecode" which
is overly complicated and at the same time inadequate for the task
at hand, much less what is coming down the road.
Ultimately, post is a database management problem, and the first
rule of database management is to have a unique ID for each record.
It seems to me that if an individual sample (in this case image)
is given a unique name, and the conditions under which it was recorded
(time, place, device, etc.) then that sample can later be synchronized
in time and space with other samples (cameras, audio recorders,
motion sensors) without ambiguity. While it may not always be possible
to record all the information one would like, time (including date)
and device (camera A) are an excellent start. Like a Kodak Keycode,
time + device provides a unique number for each frame recorded.
Unlike Keycode, useful information (time) is also embedded in the
Although Scene, Take, etc. should be recorded as well, they do not
present a unique numbering solution as does time. It would seem
more appropriate to store that information within the file itself
or as a summary of a sequence of time/device stamped frames. I personally
prefer using XML for this since it is widely accepted and can actually
be read as text, rather than binary computer speak.
Perhaps SMPTE could define a Samplecode which we could all implement
and get back to worrying about character and content rather than
dropping our frames every 1000 or so.
>all current DI vendors have
their own ways of combining this information >for the final assembly.
All file based systems have a hierarchical >directory structure
- they have to.
[Lucas] Not necessarily. Look at the way the Quantel
iQ or the Apple iPod organize media. You have the option of folders,
but much more powerful is the concept of a saveable, multi-column
metadata sort. In many projects, I've come to a point where a folder
structure that worked at the beginning of a project quickly becomes
unwieldy and/or irrelevant.
What are folders anyway? They are ways of organizing subsets of
media. If every piece of media has several metadata identifiers,
I think it's much more powerful to have scriptable, sortable searches
that can be saved. Those saved searches can be called whatever you
want -- folders, bins, libraries, etc. Look at the Smart Playlists
on an iPod. Dynamic searches that are constantly live, and filter
new material into locations based on a set of rules. Once I started
using those, when I now get locked into folders, I feel like I'm
taking a step back.
[Mike Most] To me, the complicated thing to do
is to stuff all this metadata into each and every file during live
recording (i.e., production) and make it all work in a way that
conforms to some sort of standard and is reliable.
[Lucas] There are a few basic accepted standards
of keeping track of timeline based media in post -- Keykode, Timecode,
and File Numbers. There are subsets like Edgecode, Source and Record
Timecode, etc., but every project comes back to these three basic
categories of metadata. It's a question of reliably marrying those
three metadata sets throughout the course of a project. For that,
we have EDL, FLX, ATN, FTL, AAF, AFE, ALE, CTL, and a few others
that I'm sure I'm forgetting. Each one of these has its place and
function. Manufacturers have the Keykode/Timecode relationships
pretty established. For me, the task is how do we incorporate file
names/numbers into the mix?
[Mike Most] That, to me, seems quite a ways off,
both in terms of reliability and simply deciding on a standard,
regardless of what the AAF people seem to say.
[Lucas] Call me an eternal optimist, but I don't
think it's that far off. Give me these applications: EDL Manager,
Avid Log Exchange, FilmScribe, EDLMAX24, EDL2AAF, AAF2EDL, AAF2XML,
and Excel, and I can go from just about anywhere to anywhere else
in post. Granted, that's a lot of widgets, but it's there. The stuff
we deal with is complicated. I don't think it will ever be bullet
proof, but the methods are there now. It's a question of simplification.
This is of course strictly for timeline translation. Start talking
about migrating color settings, LUTs, Color Cubes, DVEs, and things
get significantly weirder.
[Noel Sterrett] Ultimately, post is a database
management problem, and the first rule of database management is
to have a unique ID for each record.
[Lucas] Metadata is fluid. What is true for a piece
of media one day is not true the next. Maybe the timecode that was
digitized is wrong. Maybe the 16mm Keykode in FlexFile should have
interpreted as 40-count instead of 20-count. A unique identifier
on one system is not the same on another system.
There is obviously a beginning point -- the origination camera and
the format it is recording to. But from there, I think the key is
being able to fluidly navigate between list sets and metadata sets.
That ability makes translating between different systems easier,
and it makes troubleshooting much easier when something goes wrong.
(disclaimer: I work for Quantel)
Michael Most wrote :
>what's done now is to simply
name the directory that the files are in, >much as you name a
roll number on tape.
Disclaimer first - "I work for nucoda - we make visualisation
and conforming tools for the digital film market".
The concept that Michael describes above is used for what we would
call file name conform - where the name of the folder that the file
is in can be used to identify the "reel" name from an
However we also do what we call "Header conform" where
the software will look for the information specified in the EDL
- in the DPX header (i.e. timecode and reel name) The DPX header
can contain a 32 character name, which can be used to uniquely identify
every scene/take. I was actually demonstrating this to someone today,
whilst recording from a Viper.
So with every shot uniquely named, ie tk1, tk2, tk3, etc the timecode
is quite up to the task of identifying the shots - it also means
that all the frames can be mixed together in one big folder if you
want, because it is the meta-data in the frames that we read to
do the conform.
>and make it all work in a way that conforms to some sort of
>standard and is reliable. That, to me, seems quite a ways off
Works today - and the meta-data is the SMPTE DPX standard.
>So with every shot uniquely
named, ie tk1, tk2, tk3, etc the timecode is >quite up to the
task of identifying the shots - it also means that all the >frames
can be mixed together in one big folder if you want, because it
is >the meta-data in the frames that we read to do the conform.
Well, in a 2000 foot AB reel, that would mean you would have 32,000
files - and that's without handles. Seems like an awfully unwieldy
approach, and it also seems like it would take the software some
time to read 32,000 headers and build whatever database it's going
to use to index them. In any case, putting all the files in one
directory prior to a conform seems like a dubious working method,
even though the software itself can handle it. Seems to me that
separate named directories is a lot easier to keep track of and
a lot easier to deal with, particularly if those files are ultimately
going to be copied to a "conformed" directory for color
timing, either on the Nucoda system or elsewhere.
BTW, Nigel, if you could send me your contact information, I'd appreciate
it. I might need to talk to you about something in the near future.
IATSE Local 600
>So with every shot uniquely
named, ie tk1, tk2, tk3, etc the timecode is >quite up to the
task of identifying the shots
The problem with using scene, take, etc. as an identifier is that
they must be manually input and are, therefore, subject to error.
Not that they should not be included in the file header or even
the file name, but just that they are less than ideal as a unique
file ID. Device name/number + timestamp can be unique and automated
in an way that is much less prone to input error. The use of time
in this manner has proven reliable in a variety of computer systems.
The problem with using multiple directories in a hierarchical file
system is that it is often difficult to maintain the hierarchy.
If the hierarchy breaks down, which I have often seen happen, the
file names themselves are not unique and conflicts occur.
Kodak automatically generates unique Keycodes for film. Why settle
for less with digital images?
Noel Sterrett wrote :
>Kodak automatically generates
unique Keycodes for film. Why settle for >less with digital images?
True, but film is always catalogued and tracked by a combination
of key number, camera roll, and scene and take numbers, even after
it's broken down. The fact is that because film is a physical entity,
it is not possible to be "sloppy" about storage and tracking
systems and still find what you need. In many ways, it's much easier
to keep track of a physical asset than an electronic one. However,
I understand your point.
Years ago (around 1987 or so) at Lorimar we decided to try cutting
and assembling negative from video EDL's (foolish at the time, but
we did it anyway). We investigated using Aaton code as the basis
for this, figuring that each individual frame was uniquely identified
by using a combination of time code, shoot date, and camera ID.
I still think that's all you really need, provided the date and
camera ID are properly entered and the time code generator is working
IATSE Local 600
Michael Most wrote :
>True, but film is always catalogued
and tracked by a combination of key >number, camera roll, and
scene and take numbers, even after it's >broken down.
Yes, but the same can be done with electronic images, at least where
that information is embedded in the frames themselves. It would
be rather simple (in software), given even a huge single directory
to create a database cataloguing and sorting the files by any fields
(scene, take, etc.) included in the filename or file. DPX files
provide a wealth of fields to store metadata, including a "roll-your-own"
The main point is that if we insure a truly unique record ID for
frames (filename) all sorts of cataloguing and database management
systems are possible. If we don't, it's going to get very messy.
>by using a combination of time
code, shoot date, and camera ID. I still >think that's all you
really need, provided the date and camera ID are >properly entered
and the time code generator is working properly.
Exactly! This scheme works perfectly. There is one little weakness
that, I believe, doesn't play a big role in practice and can be
overcome if the software behaves intelligently: If you have a take
that wraps over midnight it might get sorted into the wrong "shoot
date" or software might have problems with the time code wraparound.
(All assuming time of day timecode).
I see your point in wanting to generate a unique id and a timestamp.
I agree that this would be desirable.- However, like I tried to
point out in a previous post: For a timestamp to work you need to
synchronize all CineRAMs on set in a perfect manner - plus you need
to synchronize them to other equipment: To tape recorders, audio
equipment, timecode slates.
There already is such a synchronization mechanism today. It is limited,
complicated, counter-intuitive etc. But it works and is the standard
We might have used CineRAMs on a current project, but it involves
4 simultaneous HD streams and can't be done without an external
The timestamp is a very good feature, because it allows you to move
the frame easily to the correct "shoot date" folder and
generally adds a searchable tag that can be used for a production
database. It also gives you a good cross check when you shoot time
of day timecode.
Still a timecode input is absolutely essential for the syncing of
several CineRAMs and other equipment.
"For years I have been struggling with "timecode"
which is overly complicated and at the same time inadequate for
the task at hand, much less what is coming down the road."
With all of the recent discussion, I will quote below, an email
I solicited from Evertz on the possibility of creating a Time Code
with Date Stamp. If any of you has questions about the process,
I am not the expert, but you can go to evertz.com and contact them
directly for further information :
"Our 5010-GPS and 5600MSC+GPS units can put the date into the
LTC user bits according to SMPTE 309M format. Our HD9010TM HD Timecode
Master unit can then take in the LTC and insert it into RP188 ANC
timecode in the HD bitstream for you to decode. Unfortunately the
HD9010TM does not lock to GPS directly...but with either the 5600MSC+GPS
or the 5010-GPS it can be accomplished .
If you are operating in a 29.97 world, then there will be an approximately
2 frame error per day. This error will get corrected by a hard rejam
to the GPS time once per day at a user specified time. So you should
always be within 2 to 4 frames worst case.
That's life in the 29.97 world! If your running at 25.00 or 30.00
FPS then there is not drift and everything stays exactly locked
all the time (except for leap seconds - where there is a jump of
GEORGE C. PALMER
HD and Digital Imaging www.hdpix.com
>"If you are operating
in a 29.97 world, then there will be an >approximately 2 frame
error per day."
That's true because the drop frame correction is not enough and
studio timecode generators related to realtime have to reset once
Maybe sometimes there will be a drop-drop timecode to upgrade from
Julian to Gregorian calendar.
Evertz TC Generators :
I work with Evertz tools all the time, they are professional studio
equipment. Good support also...
But for field use I prefer ambient products (Lockit system) because
of portability and battery power supply.
The master clock system controller ACC 101 is a powerful tool for
on set TC work. It generates, compares, reads TC in almost all formats
I can imagine on a film/HD/video set. It also allows you to connect
a GPS receiver (Garmin GPS mouse as an example) if you want to sync
the system to GPS. DCF sync and sync to different framerates as
the internal frame rate is also supported.
Chris, GÃ¼nther and their crew also have a wireless system for TC
and they support me since years.
Disclaimer: I donâ€™t work for ambient but Iâ€™m a really satisfied
I never had problems with these units, the master clock runs about
one week with a battery pack, only the lockits need more power because
of the sync generator. If you use a unit like this, you donâ€™t forget
the Ubits - better : youâ€™ll use them even more because itâ€™s so simple
to change/set them and then just go to the lockits and jam them
to the new settings. (Yes - I was asking for a wireless update of
the Ubits, itâ€™s there but nobody else asked, so they donâ€™t finish
it to a "real" product right now...) need more "askers".
Or: TriLvlSync on F900, word clock an DAT, two different frame rate
on set - no problem!
Timecode is the only worldwide standard for sync on set and a powerful
tool. If you use the genlock as well you are very accurate and wonâ€™t
have problems in post.
AND : Time of day TC needs more attention - I agree absolutely...Userbits
are more important. But I donâ€™t see a problem working with ToD,
there are simple rules like changing tape every day (if you are
shooting regular days like 8AM-8PM) and mark them; even if youâ€™re
crossing midnight you just need to look for no double TC on a single
tape...Just hire professional sound / DIT crews and it should work!
Even if you are recording rec run TC on LTC, just use one audio
track for the audio-decks ToD TC and backup with a TC Slate...
So many ways to do it - just do it in a proper way!
+++ Florian Rettich +++
+++ Europe based DIT / vision control +++