Archive for October, 2008

Shooting Pelicans

October 26, 2008

I spent the afternoon at Pigeon Point Lighthouse, practicing panning shots with my 500mm lens and 1.4x tele-extender. My subject of choice was the California brown pelican. The day was a little foggy so I’ve done a Levels adjustment on this to increase the dynamic range. By the end of the day I was definitely better at panning, though getting an initial sighting through such a large lens is still sometimes challenging. It takes practice to see an object with the naked eye and then quickly get the object in the len’s field of view–especially when that object is moving.

When shooting subjects like this, don’t forget to stop down as much as you can to increase the depth of field and improve you chances of getting good focus.


October 25, 2008

The Duck Pond, Palo Alto, CA

Digital Photographers Beware: Flash memory is not as reliable as you may think

October 25, 2008

Using flash memory cards for long-term photo storage is a really bad idea. I didn’t realize just how bad until I heard a recent talk on flash technology, which included reliability statistics for the two main varieties of flash, MLC and SLC.

Before sharing the numbers, you need to know that flash memory reliability is generally measured by two quantities: endurance and retention. Endurance measures the number of write cycles each sector of the memory can handle before serious or fatal errors begin to occur. Retention measures how long a flash memory part can be expected to hold its data reliably before attempts to read the data will fail. Of course this is all statistical, but nonetheless the numbers will give you a rough idea of the lifetimes of these memory devices.

Multi Level Cell (MLC) is the consumer-grade version of flash memory. It has an endurance of about 1000 write cycles. Think about how often you fill and erase your memory cards to decide if this number bothers you. If you are like me, you tend to keep and use memory cards for a long time, which means 1000 cycles is not a very comforting number. As an aside, I’ve noticed that my Canon G9 lets me erase a memory card with either a standard formatting operation or what is called a “low-level formatting.” Since the low-level operation takes considerably longer than the standard version, I suspect it erases every sector on the card, which imposes unnecessary wear, i.e. more cycles are consumed from the 1000-cycle budget for these memories. From now on I’ll be using the standard formatting option, which I suspect only affects metadata blocks. If I recall correctly, my Canon 10D just offers one formatting option. It is very fast, so I am guessing it does not erase every data block on the card.

Let’s now talk about data retention.

Some photographers I know have decided to use flash memory for long-term photo storage since cards are getting so cheap and the form-factor is small and convenient. Really, really bad idea. The retention statistic for MLC parts is a mere 3-4 years. And, worse, retention and endurance are not independent: flash memory that has been used for many cycles can have a significantly reduced retention time.

The characteristics of Single Level Cell (SLC) are considerably better, but you still probably should not use these devices for long-term storage. While endurance for SLC is about 100,000 cycles, data retention is about 10 years. A decade is good, but not for archival purposes. And don’t count on 10 years if you have heavily used the card—in which case, retention time can be considerably shorter.

While clearly SLC is more appealing than MLC, it is about 4X more expensive than MLC and about half as dense. I am hoping that the so-called “professional” flash cards sold as digital film are built using SLC, but I have found no way to determine that from vendor websites. If anyone has any concrete information on the products made by the major flash memory card vendors, please share.

Photography aside, I think it is safe to assume those ubiquitous USB dongles that are given away at conferences and others events are made with MLC. Keep than in mind when deciding what you store on these devices and how long you need it to be recoverable.

For an interesting discussion of retention and endurance along with several examples of how to determine suitability for use in several non-photographic circumstances, see this document titled Practical Guide to Endurance and Data Retention [PDF].

And to find out who is working to create ultra-reliable SLC flash memory parts, read this short article.

New England Foliage: Stony Brook Reservation

October 14, 2008
A Fall Reflection, Stony Brook Reservation, West Roxbury, MA

The Death of Clock Speed

October 13, 2008

Sun just introduced yet another chip multi-threading (CMT) SPARC system, the Sun SPARC Enterprise T5440. To me, that’s a Batoka (sorry, branding police) because I can’t keep the model names straight. In any case, this time we’ve put 256 hardware threads and up to 1/2 TeraByte of memory into a four-RU server. That works out to a thread-count of about 36 threads per vertical inch, which doesn’t compete with fine Egyptian cotton, but it can beat the crap out of servers with 3X faster processor clocks. If you don’t understand why, then you should take the time to grok this fact. Clock speed is dead as a good metric for assessing the value of a system. For years it has been a shorthand proxy for performance, but with multi-core and multi-threaded processors becoming ubiquitous that has all changed.

Old-brain logic would say that for a particular OpenMP problem size, a faster clock will beat a slower one. But it isn’t about clock anymore–it is about parallelism and latency hiding and efficient workload processing. Which is why the T5440 can perform anywhere from 1.5X to almost 2X better on a popular OpenMP benchmark against systems with almost twice the clock rate of the T5540. Those 256 threads and 32 separate floating point units are a huge advantage in parallel benchmarks and in real-world environments in which heavy workloads are the norm, especially those that can benefit from the latency-hiding offered by a CMT processor like the UltraSPARC T2 Plus. Check out BM Seer over the next few days for more specific, published benchmark results for the T5440.

Yes, sure. If you have a single-threaded application and don’t need to run a lot of instances, then the higher clock rate will help you significantly. But if that is your situation, then you have a big problem since you will no longer see clock rate increasing as it has in the past. You are surely starting to see this now with commodity CPUs slowly increasingly clock rates and embrace of multicore, but you can look to our CMT systems to understand the logical progression of this trend. It’s all going parallel, baby, and it is past time to start thinking about how you are going to deal with that new reality. When you do, you’ll see the value of this highly threaded approach for handling real-world workloads. You can see my earlier post on CMT and HPC for more information, including a pointer to an interesting customer analysis of the value of CMT for HPC workloads.

A pile of engineers have been blogging about both the hardware and software aspects of the new system. Check out Allan Packer’s stargate blog entry for jump coordinates into the Batoka T5440 blogosphere.

Paypal Refuses to Pay a Merchant on my Behalf

October 10, 2008

Here’s a weird one. Correct me if I’m wrong, but the primary value of Paypal is that it hides my credit card details from merchants I choose to do business with. Am I right?

Imagine my surprise when I saw the error message below when I tried to complete a recent purchase using Paypal:

Ummm. HUH? What definition of “hide” includes the vendor having any idea whatsoever what kind of credit card I’m using?? So long as my credit card is valid and acceptable to Paypal, why should it matter what kind of card it is? I called Paypal to find out what had happened.

The representative was not forthcoming, but from what she told me it sounds like Paypal has a specific agreement with this merchant and that agreement had not been updated by the merchant. She mentioned something about a commercial entity user agreement and told me she wasn’t allowed to explain further because it would expose merchant information. She suggested I contact the vendor and explain that I wanted to use a Mastercard to make a payment to them via Paypal–presumably to encourage the vendor to re-up on their Paypal contract. She couldn’t explain why this related to Mastercard in particular.

The rep agreed with my summary: That some sort of dispute or issue between Paypal and this vendor was preventing me from doing business using Paypal as an intermediary.

That ain’t right.

LISA ’08: What Every Admin Needs to Know About Solaris

October 9, 2008

Admins, fasten your seatbelts: The 22nd Large Installation System Administration (LISA ’08) Conference promises to be as jammed with useful and interesting technical content as ever and at least as much fun. Come to San Diego from Nov 9-14 to find out!

For those of you looking to dig deeper into Solaris or for those looking to understand what the fuss is all about, there is a ton of Solaris and OpenSolaris related content scheduled at LISA thanks to a lot of hard work by people both inside and outside of Sun. Here are some of the highlights.

Jim Mauro is doing a full-day POD training session. That’s Performance, Observability, and Debugging. If you only make it to one Solaris session, pick this one. Jim is a very knowledgeable and engaging speaker and the material is excellent. I enjoyed Jim’s presentation of a much compressed version of this at a recent NEOSUG meeting–it was excellent. You will definitely emerge 1) exhausted, and 2) with a much better understanding of how to use a variety of Solaris tools to solve performance problems and to better understand your systems’ workloads. Jim will lead you on a foray into the depths of the various Solaris tools that let you look at all aspects of system performance, including DTrace. Whether you are a seasoned UNIX admin, but new to Solaris, or just wondering what all the DTrace fuss is about, you will find this taste-o-DTrace pretty exciting. And if you really want to know a lot more about DTrace, Jim is also doing an all-day DTrace training session at the conference.

Peter Galvin, long-time Solaris expert and trainer and also chair of <a href="NEOSUG, and Marc Staveley will be giving a two-day Solaris workshop that has been broken into four half-days sessions. The sessions are Administration, Virtualization, File Systems, and Security. These are all hands-on sessions so Peter and Marc recommend you bring a laptop. Solaris installation not required–the instructors will supply a Solaris machine for remote access.

For something higher level and more strategic, Jim Hughes (Chief Technologist for Solaris) will give an invited talk on OpenSolaris and the Direction of Future Operating Systems. And Janice Gelb will also deliver an invited talk provocatively titled, WTFM: Documentation and the System Administrator.

There will be two Solaris-focused Guru sessions at LISA as well. Scott Davenport and Louis Tsien will cover Solaris Fault Management, while Richard Elling will speak about ZFS. These both promise to be interesting sessions with technical people who really know their stuff.

Solaris Containers are an innovative virtualization technology that is built right into Solaris and Jeff Victor will be leading a full-day workshop to take attendees on a detailed tour of this capability. Check out Resource Management with Solaris Containers.

There will also be a full-day deep dive workshop on ZFS offered by Richard Elling. Many people have heard about this new file system, but you won’t really understand exactly why it is getting so much attention until you experience how it changes the administrative experience around file systems.

Sun will also be hosting a vendor BOF to talk about BigAdmin, the mega-hub for metrics tons of useful and very detailed information for administrators. If you aren’t familiar with BigAdmin, check out the BOF or at the very least pop over to the website for a peek. Cool stuff.

Sun will also have a booth in the exhibit area. Booth 52, I believe. Stop by for some good conversation and maybe some giveaways.