Apologies to everyone who’s been eager to see the results of my Summer of Code project. School has left me busier than I expected to be.

That said, here are instructions for building render-jp2. Replace $OBJDIR and $SRCDIR with the appropriate values, depending on your .mozconfig.

1. Download and build Mozilla’s source through CVS.
2. (from the root mozilla dir:) svn co http://svn.eschew.org/projects/mozilla/jp2/libjasper/mozilla/extensions/render-jp2/ extensions/render-jp2/
3. cd $OBJDIR
4. $SRCDIR/build/autoconf/make-makefile extensions/render-jp2/
5. make -C extensions/render-jp2/

This results in an XPI in $OBJDIR/dist/xpi-stage/ named something like render-jp2.r294.PLATFORM_ARCH-GECKOVERSION.xpi.

There are also updated Firefox 3 render-jp2 builds for Mac OS X and Windows. Builds for Linux and Firefox 2 should be forthcoming.

In the last entry, I noted that JasPer itself could render all nine images correctly, but it wasn’t doing so when I was using it in render-jp2.

I dug through the sources for jiv.c yesterday, and figured out a few things I was doing differently than jiv.

For one, I was explicitly reading out the components of each image into memory separately, and then recombining the components to form the image. jiv, on the other hand, uses a different API call to read individual samples from the decoded image, without copying the samples into memory again. This seems to fix an odd and hard-to-debug error that was cropping up with images 2 and 3, where we’d try to read a bad pointer when accessing the image-width+1′th row of an image that was taller than it was wide. Very strange.

Also, jiv, unlike me, was using jas_image_chclrspc() to explicitly change the image’s color space. This fixed slightly-off colors in image 5 when I started doing it.

So now, render-jp2 correctly displays all nine images!

I don’t know how much of a performance impact these changes will have. I added a few printf()s to render-jp2, and right now it takes roughly 2.2 seconds to go from start to finish of downloading, decoding, and rendering a JPEG 2000 image in Firefox, at least for a 10 KB image. Add 0.1 seconds for an extra 100 KB. Haven’t yet tested with anything larger. This leads me to the other thing I got done this morning: Trac.

I wanted to temporarily revert the above changes to see how much of a speed difference they make. I figured, hey, I know, I’ll go into Trac and pull up the copy of nsJP2Decoder.cpp from a few revisions back. Problem was, when I visited Trac, it just spat out an incomprehensible error message. Something about “svn is not a valid repository type.” Say what? Turned out to be a combination of stale FCGI processes and missing swig-py bindings from when I recompiled Subversion on July 4th. Long story short, I spent a few hours this morning fixing my Trac install, so now there’s a web-accessible and dynamically-updated list of Subversion checkins. Much easier than running svn log…

One thing I’ve noticed: complex images aren’t very well supported by the open-source JPEG2000 decoders out
there.

For example, JPEG 2000 supports full transparency in images (like PNG, unlike GIF). However, I haven’t seen any published test files with transparency. I created one myself with the (Kakadu-based?) JPEG 2000 plugin to Adobe Photoshop. If you try getting JasPer to view an image with transparency (with four components), it throws a run-time assertion. Not good. OpenJPEG fairs slightly better, but still doesn’t render it correctly.

In the JPEG 2000 Conformance Files package, there are nine JPEG 2000 images, along with nine corresponding TIFF-format reference renderings. JasPer, being one of the reference decoders, does pretty well; OpenJPEG does not quite so well.

Key:
Y - renders correctly
P - renders partially correctly
N - does not render at all

Tests were done with JasPer 1.900.1 and OpenJPEG 1.2.

Yes, I’m still alive.

The good news is that I’ve written about 2750 lines of code in the past two and a half weeks. The bad news is that none of that has been for Summer of Code, it’s all been for my CISC 370 class: two homework assignments and an unexpectedly-large project.

Combine the time spent writing the above with the Fourth of July holiday, my girlfriend’s birthday, and twice-weekly meetings for CISC 370, and I’ve gotten roughly zero work done for Summer of Code in the last… well, since I got my wisdom teeth out, which was almost exactly one month ago. No code written, at least. I’ve exchanged email with a few people, some looking to help, others just curious about how the summer’s going, progress-wise. But that’s not what I’m getting paid for… =/

So, today I did a CVS checkout of Firefox’s trunk. It pulled quite a lot of changes, since the last time I’d synced was, oh, a month and a half ago, at the beginning of June. Problem is that doing the usual make -f client.mk build errored out in Gecko somewhere, so at the moment I’m stuck without a debug build, at least on this machine. I’ve got other builds on other machines, since I want to make completely sure that render-jp2 is cross-platform. But none of those other machines are set up to do debugging, which is what I need.

Update, 4 hours later: It worked! Clobbered tree rebuilt A-OK.

In other news, I’m back up to running 4 miles. Earlier this summer, I was training for my first half marathon, but sorta stopped when I got my wisdom teeth out. So I was back at square one, mileage-wise. Started out this week just barely running 2 miles, so 4 is actually a fair improvement. Hopefully I won’t actually die come September 9th.

Completed the Google Summer of Code Student Midterm Evaluation.

Also, cooked Vietnamese Lemongrass Chicken with Caramel Sauce the other night. Verdict: lemongrass is a bit sharp for me, but the dish goes very well with milk and spaghetti.

Combination of summer class and two homework assignments being due, Mom flying out of town (and the associated increased cooking/housework load), plus the 4th of July and my girlfriend’s birthday (celebrated twice!) means that I didn’t get to do much Summer of Code work at all this past week. Tonight I’m driving up to PA to meet with a college friend I haven’t seen in a while. Hopefully I’ll get a little bit done tomorrow (Sunday) and more this coming week. I mean, I think I’m still somewhat ahead of schedule (depending on how long item 1 below takes) but I wasn’t anticipating losing a whole week to Real Life ™. Oh well.

On my todo list is to do work on MIME type detection (chatted with pavlov and beisi on IRC, looks like this might require changing Firefox itself, not just extension code), and replace libjasper with OpenJPEG.

I did, however, get a chance to talk by email with one of the developers of the OpenJPEG library, in addition to one of the members of the JPEG committee. Not all development is code, I guess…

So the morning (and by morning, I mean after 11:15 am when I woke up… ;-)) was spent trying, and failing, to figure out why my code seemed to be choking on YCbCr JPEG 2000 images. In an image with 480 columns and 640 rows, the code was failing when trying to get the color values for the first pixel in the 480th row. And it’s not a problem with JasPer itself, so far as I can tell, because the JasPer Image Viewer (jiv), which ships with JasPer, renders the image just fine.

Sadly, however, that’s the good news. The bad news is that so far as I can tell, the state of software support for transparency in JPEG 2000 images is pretty rickety. The list of applications that claim to create transparent images is few and far between. ImageMagick says it can convert images to JPEG 2000, but converting a .PNG with transparency to .JP2 resulted in a completely black one-channel image. There’s a plugin for PhotoShop that at least outputs the right number of channels, four, but no JPEG 2000 implementation I can find, except for Photoshop itself, will actually render such files. JasPer’s image viewer hits a runtime exception in a deeply-buried assert() that makes the whole program in question come crashing down. OpenJPEG does slightly better, insofar as it doesn’t crash, but it merely renders a black square and complains “JPEG2000: weird number of components.” QuickTime’s PictureViewer, which I understand to use Kadaku underneath, and IrfanView, which uses I-don’t-know-what, both likewise render naught but a black square. Sigh.

Oh, fine, there is also slightly better news: I’ve added a little smarts to render-jp2, so now it correctly renders both one-channel grayscale and regular 3-channel RGB images, and it also fails quietly (rather than, you know, crashing) if it runs across an image that it knows it can’t render. Also, high-precision (more than 8-bits-per-component) images can be decoded amd displayed, though not with full fidelity — the extra bits are thrown away before rendering. This is because the APIs available to image decoders in Firefox, so far as I know, are constrained to 8 bits per component.

I think it might be time to consult with Stuart about which direction to head in next. After cleaning up the code I have now, I could either spend time ripping out libjasper and replacing it with libopenjpeg, which would lead to little immediate benefit, or I could turn attention to writing an image-type finder service, to complement the existing Plugin Finder Service that says “Additional plugins are needed to display this page…” when you go to a Flash page on a machine without Flash currently installed. I’m leaning towards the latter route at the moment, as I think it will have the greatest positive impact on the general user population.

I’m pleased to announce initial availability of render-jp2 installable extension bundles! Builds are available for Firefox 2 and Firefox 3 running on x86 Windows 2000, XP, Vista, and Linux. PowerPC OS X builds will be coming soon.

The XPI names are a bit daunting at first, but they break down pretty easily. For example, let’s take render-jp2.r218.Darwin_ppc-gcc3_grv1.9a6pre.xpi.

1. render-jp2 is the extension name.
2. r218 is the repository revision from whence the build came. Higher numbers mean later (and hopefully better) builds.
3. Darwin is the platform name for Mac OS X. WINNT is Windows, Linux is, er, Linux.
4. ppc-gcc3 is the type of machine the build is targeted at. ppc means PowerPC (not Intel-based MacBooks!).
5. grv1.9a6pre is the Gecko Runtime Version identifier. This indicates which version of Firefox the XPI is meant for. rv1.9 means Firefox 3; rv1.8.1 means Firefox 2.

So the above extension is meant for trunk (Firefox 3) builds running on G4-based Mac OS X machines. If you go to the page linked above, JavaScript embedded in the page will take a peek at your browser’s User-Agent string and should be able to select the right build for you. It’s not infallible, though, so if you have, say, SeaMonkey running on PPCLinux, it might advise you to install something that won’t actually work. But for most people, for the common configurations, it ought to get it right.

Anyways, these are still experimental builds, but I haven’t seen any stability issues. So please, if you’re feeling the slightest bit adventurous, install, observe, and let me know what you think!

Monday, it felt like I was spinning my wheels. I built mozilla trunk on bart, that went smoothly enough, but that was about it. Installing the XPI built on my desktop simply resulted in the console printing Failed to load render-jp2.dll — error 127. Angel refused to build the trunk, much less my extension, no matter what build configurations I tried. That was slow.

Tuesday, a breakthrough: putting $(MOZ_COMPONENT_LIBS) under EXTRA_DSO_LDOPTS meant I was using internal linkage (by linking against xpcom_core.lib), not frozen linkage! Maybe that was why it wasn’t loading? I busied myself figuring out what I could remove and what I couldn’t, but kept on running into linking issues. Unresolved symbols! PR_GetCurrentThread, PR_AtomicIncrement, PR_AtomicDecrement. After banging my head against the wall for a while and posting a query on IRC, though not in that order, a lightbulb finally turned on. I had another look at the relevant docs, and yes, that was it! I wasn’t linking against nspr4.lib, which is why all the symbols the linker couldn’t find were prefixed with PR_. Makes sense in hindsight, as things do.

Unfortunately, the net effect of this particular change was to go from “error 127″ to “error 126″. I tried looking through the source in lxr to figure out what was going on, but I hit a wall wrought from hairy platform-specific library loading calls. So, after turning to Google, I by-chance ran across a stray phrase that turned out to be just the ticket: “Try using depends.exe to figure out what dependencies your DLL has.” Another lightbulb started to glow oh-so-faintly. Subjecting render-jp2.dll to the depends.exe magnifying glass was, well, enlightening. It revealed that, duh, I’d been building a debug version of the trunk, so of course the linker was linking against the debug version of the Visual C Runtime Library (MSVCRTD), which, being not provided with the nightly builds, constituted an unresolved load-time dependency. Thus, error 126. But hey, that was steady.

After mucking about on MSDN and doing a bit of grepping through the magic that is config.mk, I saw that a combination of USE_STATIC_LIBS and MOZ_NO_DEBUG_RTL controlled the flags that direct which runtime library was to be linked against. That wasn’t quite enough, as now the linker was, by default, linking against the debug RTL, and by directive linking against the non-debug RTL. Two RTLs, two symbol definitions, one unhappy linker. So, finally, conditionally adding /NODEFAULTLIB:MSVCRTD in my Makefile was sufficient to satisfy the linker, and lo and behold, an XPI, an XPI, I do declare!

But soft! It compiles, yet will it run? A quick scp to eschew.org, a saunter upstairs, a download… and Lena, in all her DWT-encoded glory. And THAT was progress.

Now, however, I have to go eat breakfast, do errands, and drive to Newark for class. And so the adventures of yesterday must wait for tomorrow.

Long (10am to 12am = 14 hours) but relatively productive day today.

• Updated the Windows 2000 laptop (Bart) to Service Pack 4.
• Installed Visual Studio 2003 and mozilla-build on Bart, and ran initial checkout of mozilla trunk.
• Downloaded build prerequisites for a PowerBook G4 (Angel) running Mac OS X 10.3.9
• Checked out mozilla trunk on Angel, attempted build.
• Initial build on Angel choked building airbag toolkit; the machine is currently building latest attempt. CPU BOTTOMSIDE: 55 degrees C.
• Downloaded and installed VMWare Workstation 6 trial on my main machine, and Ubuntu 7.04 on that.
• Under Ubuntu, downloaded and built Minefield debug build. Interestingly, I think it took less time to install Ubuntu and get Mozilla built under Linux than it took to merely install Visual Studio 2003 on Bart.
• Went grocery shopping!

Tomorrow:

• Finish and/or confirm successful builds of firefox on Windows 2000 and OS X.
• Try prebuilt XPI on Windows 2000 debug build, see what happens
• Build render-jp2 from source on Angel and Bart
• Hopefully, end up with a render-jp2 XPI that works on vanilla Firefox nightlies.

Summer of Code got off to a slow start for me; apologies for the radio silence so far. I expected to be slightly delayed in posting updates initially; I didn’t expect it to take this long.

So today is Saturday, June 2nd. The day coding was supposed to start, May 28th, I was in Milford, DE, at a friend’s bonfire/sleepover. Wednesday was spent at Rehoboth beach (tough life, I know…), and I didn’t get back until 11pm. Thursday I did get work done, but there was a power outage Thursday night around 11:30 pm. Yesterday, Friday, was my 20th birthday. I spent the morning in Philadelphia, and then had an amazing dinner and evening with friends.

Luckily, all is not lost! I actually started working on my project towards the beginning of May, before my final exams (I was bored!). It looks like Google hasn’t had time to get the SoC-specific repositories for students up and running, so for the time being, I’m hosting my project in my personal Subversion repository.

The source code is here:
svn co http://svn.eschew.org/projects/mozilla/jp2/

The good news is that it works for me when I compile it as part of a debug build of Firefox. The bad news is that it doesn’t work (installs fine, doesn’t seem to actually do anything) when I take the resulting XPI and install it on another computer with a fresh, non-debug Firefox 3.0a6 build. And because it’s not a debug build, there’s no easy way I can figure out what’s going wrong.

So, my plans for the next few days:

• Make a debug build of Firefox on my Dad’s laptop
• Make sure the XPI is functional on vanilla trunk nightlies
• Build Firefox (and then render-jp2) on Mac OS X and Linux

And so begins Google’s Summer of Code!

My project for this summer is to give Firefox the ability to render JPEG 2000 images. At least initially, I’ll write it as an extension targeting the as-yet-unreleased code that will eventually be called Firefox 3.0. Adventurous users may install a possibly-unstable nightly build of Firefox (at their own risk!) and, when it’s publicly available, my extension. At that point, pages like Gemal’s Image Format Support test page should show a “Supported!” image in the JPEG 2000 column, like this:

More coming later, so be sure to subscribe to the feed for my Summer of Code posts , or my entire blog !