FINAL PROPOSAL: jrawio - Image I/O codecs for "camera raw" formats

Fabrizio Giudici fabrizio.giudici at
Sun Mar 2 08:52:46 PST 2008

                               FINAL PROPOSAL

Provision of jrawio, a set of Image I/O codecs for "camera raw"  
formats, suitable for inclusion in OpenJDK

Fabrizio Giudici

1. Summary

The Java Image I/O API (javax.imageio) is a specific API of the JDK  
focused on image input/output. It basically provides a plugin-based  
set of classes for reading and writing images from/to a stream (both  
raster and metadata operations are supported). In the standard Java  
runtime a number of plugins for the most common file formats are  
available (e.g. JPEG, PNG, BMP), others are in the optional package  
JAI-Image I/O provided by Sun (e.g. TIFF), others are available as  
third-parties' plugins.

With the advent of professional digital camera bodies, manufacturers  
such as Nikon, Canon, Minolta Pentax and others have introduced a  
number of proprietary formats, commonly named "camera raw" formats  
(NEF, CRW, CR2, MRW, etc...) [1]. In this scenario, also Adobe  
developed its own format, Adobe Digital Negative (DNG), which has got  
an open specification. All "Camera raw" formats share the peculiarity  
of not containing an image ready-to-be-shown, but just a "dump" of  
the raw data from the digital sensor with a set of annotations about  
the camera settings at the moment of the shoot. This makes them a  
sort of "digital negative", as photographers can manipulate  
("develop") them in post-production with a much wider gamma of  
possibilities than JPEG or TIFF would offer; it also makes it easier  
to implement "non-destructive processing" (the capability of undoing  
a developing step without compromising the original bits).

jrawio [2,3] is an open source implementation of a set of read-only  
plugins for supporting these formats.

jrawio is an original development. At the moment of writing there are  
no other known Image I/O-compliant plugins for "camera raw" formats.  
jrawio only uses original code (of course integrated with the Image I/ 
O API, which implies that jrawio implements some of the interfaces  
defined by Image I/O). jrawio is partially inspired by dcraw.c by  
Dave Coffin [4] as most of the "camera raw" formats are undocumented  
(DNG being a notable exception) and dcraw.c is the only available  
"documentation" for the community. An older dismissed project by  
Fabrizio Giudici, NEFio [9], was just the initial version of jrawio  
with a different name as it only focused on the NEF format.

The goal of this proposal is to complete the work required for making  
jrawio a part of the OpenJDK.

This project is a single-developer submission. Fabrizio Giudici is in  
charge for the rules of the Open JDK Challenge.

2. Need for jrawio

While most of “camera raw” formats are a variant of TIFF, they can't  
be properly handled by the existing TIFF Image I/O plugin, because:

1)	it can't understand the proprietary extensions, both in metadata  
and raster layout, also because often non-standard compression  
schemata are used
2)	as all the other Image I/O plugins, it doesn't perform any post- 
processing on the raster data

The typical post-processing required by “camera raw” formats is  
composed by:

1)	demosaicing
2)	noise reduction
3)	boosting of R-G-B channels
4)	applying a color profile
5)	sharpening

3. Reasons for having jrawio in the OpenJDK

We believe that support for “camera raw” formats should be present in  
the OpenJDK, and not as a separate extension, since these formats are  
more and more used by advanced amateurs and professional  
photographers in place of JPEG and the ubiquitous availability of  
media codecs out-of-the-box is important for Java to be able to  
compete with other technologies for the desktop such Adobe Air. Also,  
a small but specific part of the post-processing (the demosaicing  
step) could benefit from a small native accelerator, whose presence  
in the Java runtime would make easier the life for Applet and JavaFX  
developers. Last but not least, many communities of photographers  
(see e.g. OpenRAW [10]) have publicly stressed the importance of  
having open source implementations of software for managing "camera  
raw" formats; the presence of out-of-the-box codecs would make Java  
more friendly and appealing for developers targeting the imaging  

4. Structure of jrawio

jrawio is composed by two parts:

a) the javax.imageio plugin which only delivers the capability of  
reading metadata, thumbnails and "raw" bits out of the stream;
b) reajent, the post-processor that "develops" the raw bits into a  
viewable picture, implementing the typical operations described in  
the above section. This is an optional component, as applications  
could decide to provide their own implementation of this part  
(providing special "recipes" for developing "camera raw" formats is  
typical of this context).

5. Previous work

Work on jrawio and reajent has already started [2,3] and a 1.0.RC5  
has been delivered which is reasonably stable and suitable for  
experimental use, but not for production. Since 2007 the focus has  
been on the setup of a public test suite - some required work and  
technical problems about tests has slowed down the progress and  
therefore there have been only a few unofficial releases in the  
latest year.

The submitted project represents the continuation of that development.

At the moment, the metadata and raster I/O parts of jrawio support  
the following formats: CR2, CRW, DNG, MRW, NEF, PEF, SRF, but only  
for a subset of the existing camera models (each camera model is  
likely to use a variant of the main format). It is estimated that the  
current codebase contains about 80% of the needed classes for those  
formats, that should be complete at 70% (as per the number of camera  
models existing at the moment of writing). Support for other formats  
(such as Kodak DCR) should be added, but it's difficult to estimate  
the effort for them without prior knowledge of their structure.

Unit tests exist for about 10% of the codebase, while there are  
integration tests over a set of around 1,000 image files verifying  
that jrawio correctly reads rasters, thumbnails and metadata. Beyond  
the already mentioned technical problems specific of the test code, a  
licensing problem exists about the test set since it's mainly  
composed of images available on the Web, with no permission about  
redistribution. The "Imaging Test Set Repository" initiative [5]  
should give a solution to this problem.

The jrawio and reajent implementations are licensed under the Apache  
Software License 2.0. We are open to change it as needed for  
compliance with the OpenJDK licensing.

If this proposal is selected, and proceeds to completion, the entire  
project will be made available under the SCA (both the parts already  
coded, and the additions made since).

6. Milestones

Development milestones shall ultimately be determined by the working  
group. However the outline plan is:

1) Provide tests and appropriate bug fixes to verify the full  
compliance with the javax.imageio APIs

2) Add support for all the camera models using the currently  
supported formats

3) Provide a small extension mechanism (probably based on META-INF/ 
services) so support for new camera models requiring specific code  
can be just dropped in as a classpath extension. This is needed since  
new camera models using "camera raw" formats are released much more  
frequently than JDK updates and application developers typically want  
to stay up to date with the shortest latence.

4) Add support for new "camera raw" formats (e.g. Kodak's DCR) as  
long as their structure is understood.

7. Deliverables

The deliverables are:

1) The source code of jrawio (both the jrawio plugin and the reajent  

2) Tests with 90%+ coverage.

3) Source code documentation in the form of javadoc suitable for  
inclusion in the JDK.

Any API extension for dropping in custom-specific reajent  
replacements does not form part of the deliverables of this proposal,  
since it won't be possible to complete this task by the August  
deadline of the OpenJDK Innovation Grant Challenge (considering that  
it would require a prior submission to JSR etc...). By that time  
programmers will be able to just disable reajent (by means of the  
existing javax.imageio API) and will manually post-process the raster  
at application level.

8. Dependencies on Sun

There are no known dependencies on Sun - the project just supports  
javax.imageio and will be kept up to date with the evolution of those  
APIs. While jrawio contains some TIFF-related parts, it is completely  
independent of the JAI-Image I/O TIFF plugin, both at compilation and  
at runtime stages.

9. Developers

Fabrizio Giudici has been working with Java since 1996, from J2ME to  
J2EE. Fabrizio has been the designer and the project leader in a  
number of industrial projects including banking services, support  
systems for telecommunications and a real-time distribution  
middleware for telemetry data in Formula One racings. He currently  
works for his own company, Tidalwave s.a.s. [8], as a Senior  
Architect, Mentor and Technical Instructor. Has has developed in Open  
Source since 2003 as the author of jrawio, Mistral [7] and blueMarine  
[6]. He is a Member of the NetBeans Dream Team since 2007.


Fabrizio Giudici, Ph.D. - Java Architect, Project Manager
Tidalwave s.a.s. - "We make Java work. Everywhere." -
Fabrizio.Giudici at - mobile: +39 348.150.6941

More information about the challenge-discuss mailing list