I wrote this document in February 2007, as a coursework during the last year of my French engineering school. It's entitled "OK, Houston, we've had a problem here.", in reference of the problem that concerned the Apollo 13 mission, and
the problem of the NASA tapes, nowadays. The aim was to write about 3000 words about an Internet related topic. I chose the formats because, as a regular reader of the
formats-ouverts.org blog, I believe in their importance in this field, and I wanted to share these information with my classmates. I got a lot of information on this blog, but not only (cf my sources at the end).
Introduction
Any information needs a protocol, a format, to be read and
understood. If you read this text, you can understand the English
language. If you make a phone call, your phone can transform voice in
electric signals. The Nü Shu writing system is a Chinese system
derived from Mandarin only used by women. It is also almost
completely unknown, because its last proficient user died in 2004.
Because it remained secret until 1995, and because it was customary
to burn writings during the author's funeral, there are a few Nü
Shu manuscripts left to help us understand this system. This kind of
issue is possible in the computer field. If your favorite software's
company decides to stop its development, or, worse, closes, all your
documents created with this software are lost.
These examples illustrate the importance of the formats. Formats
are important because they are everywhere. They are important because
they are the key to the access to information. An open or standard
format is necessary when you seek interoperability, data
conservation, or security. A closed (often proprietary) or
non-standard format can lead to the problems we just saw, loss of
information, control of the user or balkanization of a specific
field. That's why their choice should not be neglected. In France,
since June 2004, a law defines what is an open format, and the
government, providing the general repository of interoperability,
tries to promote the use of open formats in its administration.
Unfortunately, people are unaware of the problems that can occur
because of a wrong format. Several organizations (World Wide Web
Consortium, etc) and Web sites (http://formats-ouverts.org, etc) try
to show that formats are important, but companies with their
economical goals are engaged in a war of formats.
Importance of formats
Definition
Let's define what a format is. Wikipedia gives this definition on
its Web site (translated from French) [WIK02] :
“A format is the way (eventually normalized), in the computing
field, to represent data with binary numbers. A format can be a
protocol, a file format, a convention. It can be proprietary, open,
standardized. Standard formats are used when interoperability between
different software is aimed.”
A format can also define the type of connector (e.g. USB), the
character encoding (e.g. ASCII), etc. Formats are important because
they are everywhere. They were everywhere before the development of
computers, and were already creating problems. The most common format
is probably the language we use to communicate. If you can read this
document, it's because you understand English. But how can we read a
text if we can't understand it's language ? And what if nobody can
understand it ? That's the problem with the Phaistos disc [WIK03],
which, unlike the Rosetta stone, contains writings in only one
language. As we have nothing to compare the disc with, the language
can't be understood.
The source of
many problems
This kind of problem also exists in the computing field. It is
impossible to open a document if we don't have a software that knows
how to decode it. This is just the source of another issue. In order
to open a certain file, we need a specific software, sometimes in a
specific version. To run this software, we need a specific operating
system, which runs on a specific type of computer. We have here a
chain of formats, that can control and limit the user in its choices
and its freedom.
The domination of Microsoft and Intel in the computing field is based
on this. Lots of files are exchanged in formats that are readable by
softwares that exist on Microsoft's operating systems only. Such
systems can be installed on Intel (x86) computers only. In order to
read this file, you need to respect all these constraints. The
computing field is not limited to file formats. For example, the
Skype software, which allows to make phone calls over the Internet,
uses its own protocol to communicate, hence it is impossible to make
it communicate with other softwares.
What if real phone calls couldn't be made between two different
telephone operators, or telephone brands ?
An important
choice
Let's now consider a document provided by a government to its
citizens. Any citizen should have access to this document, regardless
of the format in which it is written. This means, being able to read
it from any type of computer, with any operating system and any
software. This is interoperability. This also means being able to
read it in 10 or 20 years. This is data conservation. The French
radioactive waste management centers have to store their data during
300 years. To do that, they have chosen a special paper and a special
ink. Digital formats weren't a solution because of the problems we
just talked about. They would have required the conservation of all
the elements of the chain : the software, the operating system and
the computer. We will see that governments and public administrations
are considering these issues and are publishing recommendations, and
sometimes laws, to favor the use of open formats.
Choosing the right format
Five questions
Now that we know that formats are important, we need to know how to
choose them. We will see that choosing the wrong format can lead to
loss of information, or control and limitation of the user. On its
website dedicated to open formats [FOO04], Thierry Stoehr explains
that we should ask ourselves 5 questions before choosing a format :
Are the file format, protocol or programming language open ?
Are the metadata (author, title, date, etc) open ?
Is the license (terms of use, etc) soft concerning the user ?
Are the hardware (computer, etc) and the physical support (cdrom,
etc) durable ?
Is the document's follow-up accessible ?
These questions can ensure that your information will be accessible
in time and from any platform.
Governments can
help
Since information provided by governmental services must be
accessible by everyone, several governments have voted laws that make
the use of Open Format Document (ODF) mandatory for office documents.
ODF is an OASIS standard approved by the ISO as an international
standard. In France, since June 2004, a law defines what an open
format is [LOI00] :
“Is considered an open standard any communication,
interconnection or exchange protocol, and any substitutable data
format which technical specifications are public and free of access
or implementation restrictions.”
France, Norway, Belgium, Denmark and the state of Massachusetts
also recommend open formats for their applications and documents [LMI00]
,[FOO02], [FOO03]. Administrations seem to follow these
recommendations, the French ministry of agriculture, the national
police, and the ministry of finances have chosen the Open Office
software to edit their documents. ODF being the default file format
for this application, this ensure that any other office software can
implement it in order to read ODF documents. Other administrations or
cities (Amsterdam, Munich) in the world are also interested in
reducing costs, ensuring data conservation and interoperability by
choosing open formats and free softwares.
Some people even go further with the definition of an open standard.
The European Interoperability Framework for pan-European e-Government
Services gives this definition [EIF00] :
“The following are the minimal characteristics that a
specification and its attendant documents must have in order to be
considered an open standard :
- The standard is adopted and will be maintained by a not-for-profit
organization, and its ongoing development occurs on the basis of an
open decision-making procedure available to all interested parties
(consensus or majority decision etc.).
- The standard has been published and the standard specification
document is available either freely or at a nominal charge. It must
be permissible to all to copy, distribute and use it for no fee or
at a nominal fee.
- The intellectual property - i.e. patents possibly present - of
(parts of) the standard is made irrevocably available on a
royalty-free basis.”
Compatibility
against interoperability
Another difficulty when choosing a format is that compatibility and
interoperability mean the same for a lot of people. Compatibility is
when a software understand a format (or protocol) that is not its
default one, e.g. Apple Pages 2 is compatible with the Microsoft Word
(.doc) format. Interoperability is when several softwares understand
each other because they use the same open format (or protocol), e.g.
WengoPhone and BeWip are interoperable because they both use the SIP
protocol.
- Compatibility between formats
- Compatibility with a de-facto standard
- Interoperability
Illustrations by Camille MoulinNow that we've established that we should use open standards to store
our documents or to communicate, let's see a comparison of open
standards and closed standards. A detailed list can be found on
Wikipedia [WIK00], although it only concerns file formats :
|
Functionality
|
Closed standard
|
Open standard
|
|---|
|
Formatted text
|
Microsoft Word (.doc)
Apple Pages (.pages)
|
Open Document Text (.odt)
|
|
Spreadsheet
|
Microsoft Excel (.xls)
|
Open Document Spreadsheet (.ods)
|
|
Image
|
Tagged Image File Format (.tiff)
|
Bitmap (.bmp)
|
|
Lossless audio
|
WAVEform (.wav)
|
Free Lossless Audio Codec (.flac)
|
|
Instant Messaging
|
Microsoft Messenger
|
Jabber
|
|
Voice over IP
|
Skype
|
Session Initiation Protocol (SIP)
|
Table 1:
List of closed and open standards
Several organizations contribute to create standards. The most known
are :
The Internet Engineering Task Force (IETF) which publish Request For
Comments (RFC), for example the SIP RFC :
http://www.ietf.org/rfc/rfc2543.txt
The Institute of Electrical and Electronics Engineers (IEEE) which
publish norms, for example the Ethernet IEEE 802.3 norm.
The
International Organization for Standardization (ISO) which
standardizes norms, such as ISO 15836, the
Dublin Core metadata element set.
The World Wide Web Consortium (W3C) which publish recommendations to
promote compatibility between World Wide Web technologies, such as
XHTML.
A detailed list of organizations can be found on Wikipedia [WIK01].
It's risky
We've all heard about the several delays in the Airbus A380
production. These delays are caused by the complex organization of
the several factories in France and Germany. Whereas German factory
of Hamburg uses paper, French factories use a specific software to
manage their productions. This software was finally adopted by
Hamburg factory, but needs a long training to be efficiently used.
Meanwhile, the project is more and more late. Second, when French and
German both use a software to conceive electrical plans, they are not
the same and, of course, not compatible. So, choosing a wrong
software, therefore the wrong format, is not without risks.
Incurred risks with closed formats
User controlled
and limited
There are several risks of choosing a closed format. The most
important one is probably the fact that a user, when his documents
are encoded with a closed format, is controlled and limited in its
freedom. Anyone should be able to change of operating system or word
processing software freely. On the other hand, anyone should be able
to read a document sent to him without having to purchase the
specific software. Roberto Di Cosmo, researcher in computer science,
explains [COS00] Microsoft's strategy to force its users to purchase
regularly its softwares. How do they manage to control the user this
much ? They make impossible for a former version of a software, let's
say Word, to read correctly a document created with the latter
version. For example, it's hard, even impossible, to open a Word 2003
document with the Word XP software. If this situation comes up, you
will be forced to purchase the latest version of Word. So you're
controlled because you are forced to buy the latest version of your
software(s) regularly, and you're limited because if you want to
change of software, you have to find another software compatible with
your documents. Lots of softwares are compatible with the Microsoft's
file formats so the limitation is not that strong, much less are with
the Apple Mail.app's format (.emlx) [PIL00].
Balkanization
Another problem caused by closed formats is the balkanization of the
markets. In this case, balkanization means the subdivision of a
market by several vendors, each one promoting its product, base on a
closed format. The best example is the music market, with its
particularities : on-line stores, Digital Rights Management (DRM) and
mobile players. This market is led by Apple with its iTunes platform,
and Sony with its Connect platform. The problem comes from the
formats of the music sold on these platforms : closed, unique for
each vendor, and incompatible with each other. Furthermore, music
bought on iTunes can't be played on a Sony player, and vice-versa.
This tends to split the market and to loose the consumer. This
problem already happened a few years ago when the Web was divided in
two : one for Microsoft and one for Netscape. It was solved because
they both made an effort and followed the standard recommendations
published by the W3C. Hence, the Web is now a place where almost
every website is visible from almost any browser, on any platform.
Not all websites from all browsers because some websites remain
accessible from certain browsers only, like Connect which requires
Internet Explorer 5.5 and more.
Hackers to the
rescue
Closed
formats are also the reason why some people crack softwares. The
famous DVD Jon, which cracked the protection on DVDs (Content
Scrambling System, CSS). Any player capable of playing a DVD must
implement this protection system. Unfortunately, Linux users couldn't
play their DVDs because no software implement the CSS protection.
Using this crack, and of course creating it, makes the user an
outlaw, because of the Digital Millennium Copyright Act (DMCA) in the
United States, and its equivalent EUCD in Europe (DADVSI law in
France). DRM, which are partly in cause in the balkanization of the
digital music market, are also the reason why some developers create
cracks such as DRM Dumpster, or FaireUse4WM. This is not the only
case where users must brake the law to be able to use something they
legally bought. Reverse engineering is a method that consists in
analyzing how something works (a software, a codec, a protocol, etc)
to recreate it. The French law authorizes this under certain
circumstances, but when the system is protected by a patent, it
becomes illegal. The Skype protocol was recently cracked by a Chinese
company [PAG00], after a reverse engineering operation. Since the
protocol was not patented, yet kept private, this is not illegal.
Nevertheless, when it comes to personal information, I think
performing such operations is justified in order to know what is
going on inside.
Conclusion
We've seen that formats, if they are not considered as an
important parameter in your choices, can lead to bad and sometimes
irremediable problems. We explained why such problems can happen,
whereas consumers are expecting an easier way to work and to
entertain using computers and digital contents. Some firms take
advantage of these expectations to make profit, and to abuse the
consumer.
The best way to prevent himself from such issues is to get
information about the importance of the choice of a computer, its
operating system, the softwares and the documents you use.
Information that can be provided by international organizations, Web
sites, and even by local associations.
The message that tries to give this work is not to necessarily
use open formats and standards, but to consider them when you seek
interoperability between different softwares, long conservation of
your data, etc. This is just easier to achieve with open formats than
with closed. Furthermore, any document needs to be stored, and the
support is also very important. Meanwhile, the war of formats
continues.
Bibliography
/ Webgraphy
WIK02:
Wikipedia, Format de données, 2007-01-31
WIK03:
Wikipedia, Phaistos Disc, 2007-01-30
FOO04:
Formats Ouverts, Pour les formats ouverts,
LOI00:
Legifrance, LOI n° 2004-575 du 21 juin 2004 pour la confiance
dans l'économie numérique, 2004-06-22
LMI00:
Le Monde Informatique, France: un rapport gouvernemental soutient les
standards ouverts, 2006-10-04
FOO02:
Formats Ouverts, Une nouvelle prise de position importante du
Massachussets, 2005-09-02
FOO03:
Formats Ouverts, Le Danemark déclare qu'il va utiliser le
format ouvert ODF, 2006-05-26
EIF00:
European Interoperability Framework, EIF For Pan-European eGovernment
Services, 2004-11-01
WIK00:
Wikipedia, Alternatives ouvertes aux formats fermés,
2007-01-29
WIK01:
Wikipedia, Standards organization, 2007-01-08
COS00:
Roberto Di Cosmo, Le Hold Up Planétaire, 2006-08-31
PIL00:
Mark Pilgrim, Juggling oranges, 2006-06-16
PAG00:
Charlie Paglee, Skype Protocol Has Been Cracked, 2006-07-13
Table
of figures
Illustration 1: The
chain of formats
Illustration 2: Five
questions
Illustration 3:
Compatibility between formats
Illustration 4:
Compatibility with a de-facto standard
Illustration 5:
Interoperability
Table 1: List of
closed and open standards
Illustration 6:
Balkanisation of the music market
Screenshot 1: Fake
Skype (left) in action
Un buzz incroyable a été généré autour de l'iPhone depuis le son annonce en janvier par le gourou Apple, Steve Jobs. Le résultat est là, 525 000 unitées vendues le même week-end de son lancement et l'action Apple (AAPL) culmine une nouvelle fois à un record historique de 132,33$ à l'heure où ces lignes sont écrites. L'action continue de prendre ses envolées lorsque des rumeurs d'une version nano et/ou GPS de l'appareil font surface ci et là sur la toile.
Jamendo continue sur la voie du succès. La plateforme d'échange de musiques gratuite gagne en popularité auprès des particuliers mais surtout auprès des artistes qui n'hésitent pas à publier gratuitement des albums complets. 
branchez le sur une prise secteur et sur votre ordinateur et hop, vous êtes prêt à surfer en haut-débit. Exit donc les installations par un technicien, les délais d'activation de votre ligne ADSL et les autres problèmes dus au côté materiel de la ligne.
