Thursday, June 17, 2010

Playing with filters

I was up in Russian River a few weekends ago on a lazy afternoon, boyfriend snoozing on the couch. I had just bought a bunch of filters for the camera; they're cheap relative to other camera accessories, and I thought they might be fun to play with. Here's a shot of the view from the deck with no filter:
First, linear polarizer (the GF1 does contrast auto-focus, so you can use them, and this was focused to infinity anyway):
Rotating the linear polarizer seemed to have no effect, which isn't what should happen, AFAIK. Switch to the circular polarizer (which is really a linear polarizer with a scatter filter behind it). Rotating this does something; here are the two extremes:
Note the emphasis on the sky in the left image and on the trees in the right (light polarized differently from each?). The left is 1/2500s, right is 1/1600s (less light coming from the trees than the sky).

Next up: stacking a linear polarizer in front of the circular one (light -> linear -> circular -> lens). In theory, this creates a variable neutral-density filter controllable by the alignment of the two polarizers. However, the alignment of the front polarizer against the outside light matters too, so it's not perfectly "neutral" (I think that's responsible for the blue tint below). Extremes:
These are auto-shutter-speed, so the significance of the difference isn't apparent: left is 1/1600s, right is 1/500s (and still much darker).
Next up: colors. As you would imagine, these make the image...colored. If you then desaturate, though, you can focus on different things. mikeyd points out that you can get almost the same effect digitally, except I might be able to pick up a bit more detail by chopping out the light I don't want before it gets to the sensor.
Yellow (nice glow along the horizon):
Red (very flat; not much anywhere near red in this image):
Blue (totally blown out from the sky, but you can see shadow detail that you can't see in other images here):
Green (similar to yellow, with a bit more sky):
And then we fall off the visual spectrum and into IR, relying on the sensor to pick up non-visual light, even past its built-in hot mirror.
Near IR (starting to lose the sky):
Deep IR (sky gone, significant shadow highlights):
These take patience and a tripod, and you can't shoot action; the first is 1/6s and the second is 8s (yes, 8 seconds).
List of filters used here:

Saturday, April 10, 2010

Home video re-think

Choosing a platform


There's no shortage of alternatives to the traditional cable box + TV model, from cable provider DVRs to TiVo (yes, people actually still own those) to more obscure offerings like Myka, or MythTV running on your closest whitebox. However, if you want to combine easy content sourcing, central storage/management with streaming, a nice remote control interface and solid, attractive hardware, there's really only one option: for better or worse, Apple's iTunes/Front Row.

Electronics


I already had an iMac that had ended up at a common-area computer desk in my apartment. This seemed a reasonable choice for a media server, though I suppose I could've shot for something that had a concept of running headless (another Mac Mini).

Output streams to the TVs had two options: the Mac Mini, or an Apple TV. The Apple TV is smaller and cheaper, but you're locked out of all the real computing functionality (the LAN-party-on-TVs option disappears). It would be nice to be able to control these with just Apple's media remote, but it's just not practical; OS X is interrupty as hell, with updates and license agreements splattered all over. The diNovo Mini makes a decent keyboard/mouse combo that you can stuff in a drawer, but don't expect to game on it.
As actual displays, I went with LED-backlit Samsung LCDs, for the lower power usage and the light weight for wall hanging. Add in some cables and we're good to go...except that it's all sitting on the floor.

Wall mounting


Fortunately, my two TVs were wall mount efforts #6 and 7 for Kacirek, so this went really smoothly. I picked up wall mount kits from Monoprice. In short: stud finder, level, pilot holes, lag bolts, bolts to the TV, hang, done. There are even nice wall mount kits for Mac Minis, naturally at more than twice the price of the LCD mounts, since they count as "designer". The Mac Mini power adapter fits really nicely in a cable-management cutout at the back of the TV. Add in an extension cord and some cable management from Fry's, and voila:



Front Row love and hate


Front Row is, at times, awesome. It remembers pause position across different streaming clients. The interface is simple and useful. Over a fast network, seeking and fast-forward are lightning-quick. It doesn't let you set a default streaming source, but that only adds a couple of clicks.
Sometimes, it's horribly frustrating. It hangs indefinitely and uninterruptably trying to load remote library contents. It forgets pause position, even on the same machine. None of these are repeatable, so trying to solve them seems nigh-impossible.

Unofficial content


iTunes also doesn't want you using their fancy toys with torrented files; it won't let you add them to your library, and if you change the file type to work around that, it still won't stream them to remote clients. Fortunately, this is pretty simple to work around. You need Quicktime Pro, which comes with Final Cut Studio, is cheap to buy separately, or can be obtained by whatever means you like. It hides in Utilities once installed, and is easy to confuse with Apple's stripped-down but base-install Quicktime Player. Follow steps 1-4 here, and your torrented content is now draggable into iTunes and streamable to Front Row clients. It doesn't re-encode unless you do steps 5-8, so it's fast and you don't lose quality.

Automatic wake-up


I also wanted waking up the Mini clients to automatically wake up the iMac file server, so I didn't have to leave it running all the time. Again, this isn't too hard. First, pick up SleepWatcher, clearly written and packaged by someone who's never heard of dmg or a Makefile (but it works). Install wakeonlan, a tiny little PERL script that sends Wake-on-LAN magic packets. Then, add something to /etc/rc.wakeup like:
COUNT=0
while [ ${COUNT} -lt 10 ]; do
/usr/local/sbin/wakeonlan 00:23:df:fe:c6:6d
COUNT=$((COUNT + 1))
sleep 1
done
Your path to wakeonlan, MAC address (of your fileserver) and packet count (and time) required for your network interface to come online may vary.

Hello, iPad?


It would be really great to be able to pull a Minority Report-style video transfer, moving streaming video seamlessly from a TV to the iPad and walking away with it. This is a pipe dream, however, until Apple decides to actually support streaming on the iPad. Seriously, Apple, I have to plug this thing in and copy the whole video to it to watch it?

Wednesday, April 7, 2010

(Wireless) network optimization, 2010 edition

After getting the Internet connection all tuned up, it's time to talk network speed.

How fast do you need to go?

Talking about gigabit speeds around the office drew some incredulity. Most users seem to be used to talking about Internet connection speeds in the sub-10mbit range, so a 10mbit hub (which my new apartment came prewired with) and 802.11b (6mbit/s TCP) or at least 802.11g (~30mbit/s TCP) will pretty much suffice. Political arguments about the mess that is US last-mile Internet connections aside, however, there are expensive options at higher speeds. Some areas have FiOS (though Verizon has apparently stopped rolling it out), and Comcast has a 50mbit/s "Extreme" plan in my area for $115/month. DOCSIS 3 supports up to ~160mbit/s link speed. Broadband speeds don't obey Moore's law (mostly due to the enormous infrastructure investment required to deploy new tech), but we'll still probably see cable plans breaking the 100mbit/s barrier in 3 or 4 years max. In short, it's a gigabit ethernet (~700mbit/s in real life) and 802.11n (~150mbit/s with today's gear) world for the short- to medium-term.

Defining your users

My use cases for wireless at home divide pretty neatly into two categories: high-bandwidth, low-latency streaming to fixed points (Mac Minis hooked up to my TVs), and bursty-bandwidth, can-survive-momentary-latency clients that move around a lot (laptop, cellphone, iPad). I'd like both to be able to max out my Internet connection, but the video streaming needs to be able to do better inside my network (streaming from my iMac). This may get murkier once Apple get iTunes streaming to the iPad working.

The first hop

No amount of optimization on the wireless side is going to help if the cable modem to router hop can't push the full speed of the 'net connection (this presumes that they're two different devices for you). First, both should support GigE (I have a Motorola SB6120 and a D-Link DIR-855). It's harder than it should be to verify the connection speed between these two; in the end, I had to force the link on the router end to 1000mbit/s-only, then make sure it still connected.
My apartment isn't wired ideally, so the cable modem and router are in different places. The apartment has ethernet throughout, but it's only wired with 2 pairs (out of the 4 pairs in an RJ45 connector); that's only sufficient for 100mbit/s links. Kacirek brought over the toolkit and we appropriated some telephone wiring to serve as the extra two pairs, replacing the 10mbit hub with an ethernet coupler.

Going N-only

The 802.11b to 802.11g migration was a mess; networks effectively dropped back to all-B in the presence of even a single B device. G to N isn't as bad, but it's not great; 802.11 continues to accumulate backwards-compatibility hacks all over the place. However, I was surprised to find that every device except my old T60 supports N, including my Nexus One. It didn't ship with the support, and Google never indicated that an upgrade to it was forthcoming, but it must have snuck out with a firmware upgrade somewhere. After digging out an old 802.11n mini-PCI card that I bought years ago and upgrading the T60, I was able to switch from G/N to just N. This is probably a significant win, if you can manage to upgrade all your devices. If not, confining the older ones to 2.4GHz (leaving 5GHz to pure-N, rather than A/N) is probably your best bet.

There's N, then there's N

802.11n has to be one of the most consumer-confusing specs ever. N works by using multiple antennas to build virtual "spatial streams". For example, radio one has antennas 1A and 1B; radio two has 2A, 2B and 2C. The silicon supports 2 spatial channels, which get built between, e.g. 1A and 2B, 1B and 2C. These spatial channels are treated as separate links, even though they're on the same frequency. There are 16 possible antenna/radio configurations and 30 antenna/radio/spatial channel configurations. The configurations are abbreviated AxB:C (A: transmit radios/antennas, B: receive radios/antennas, C: processor-supported spatial channels). The spec goes up to 4x4:4. Unfortunately, this means that 3-antenna systems aren't necessarily 3-stream (and most sold today aren't). You can't have more streams than your lowest radio/antenna count, and your maximum speed is determined by your number of streams and frequency width. N can use 20MHz or 40MHz of radio spectrum. The DIR-855 I bought seems to be either 2x3:2 or 3x3:2; 300mbit/s max at 40MHz. It seems to be impossible to buy 3- or 4-stream consumer gear at the moment (and you need client gear to support it, so it wouldn't be too useful).

2.4GHz vs. 5GHz

802.11n makes the frequency choice even harder than it used to be. 2.4GHz is an overpopulated ghetto unless you live on double-digit acreage. It penetrates walls significantly better than 5GHz, but that's a blessing and a curse: you can use it from further away, but your neighbors interfere from further away. Even worse, at 40MHz, 802.11n takes 2 of the 3 non-overlapping 2.4GHz channels. That means that if you can see two or more neighboring access points, you're not getting full speed. The penetration advantages are significant, though: my iPad gets 6mbit/s link speed on 5GHz at the furthest point in my apartment from my access point. At 2.4GHz, it gets 26mbit/s.
Dual-band solutions help, but you have to be careful. Assign different SSIDs to your 2.4GHz and 5GHz networks, so you can force clients to one or the other. Put things like video streaming in 5GHz, where a neighbor download isn't likely to cause hiccups. Test your other devices at maximum range, and see whether you can live with the 5GHz signal level.

A little more range

If you'd like to squeak just a little more range out of your access point, either to be able to use 5GHz where you would've used 2.4GHz, or to be able to reach far-away spots with anything at all, consider replacement antennas. Higher-grade access points support them, and they'll buy you a little bit, though don't expect miracles. I picked up 3 of these, which help a bit without taking it to ridiculous extremes.

Other optimizations

Location, location, location: put your access point in the middle of your coverage area. It's the simplest thing you can do to get massive speed gains.
Make sure 802.11n short guard interval is enabled: without it, you lose about 10% of speed.
Plug it in: if you can wire it, do. This is especially true for things like home fileservers, as wiring things that you copy from to wireless effectively doubles your transfer speed and halves your latency.
Screw the neighbors: some access points have options to disable being friendly to other 802.11a/b/g devices.
Disable all the complex router filtering: turn off web filtering, try things without QoS, disable the "firewall" and "security" features. Usually processors in routers are pretty light-weight, and you can be wasting CPU time on faux-security that could be spent moving more packets faster.
Fix DNS: good chunks of interactive browsing time is spent waiting for DNS servers. Your router may be doing DNS relaying (running a local DNS server); try with and without it, to see if it sucks. If your ISP's DNS servers suck, try Google Public DNS; it's further from you network-wise, so it can't possibly be as fast as your ISP's DNS servers could be, but it's quite likely faster than they actually are.

Monday, April 5, 2010

Comcast hacking

I finally gave up on my principled stand against traffic shaping and switched to Comcast when I moved. The 10x speed for the same price over Speakeasy might have had something to do with it. It's been an interesting road getting it working as I wanted, and here's the notes list:
  • D-Link modems don't work (well, at least). The one I got needed a custom firmware to work with Comcast according to D-Link's website, and the instructions for flashing the firmware included running a local TFTP server, then telnetting to the modem. Connection refused. Classy, guys. Took it back to Fry's, noticed that every single D-Link and Linksys cable modem box had a returned sticker on it. Bought a Motorola SB6120.
  • DOCSIS seems to support remote firmware flashing. The modem comes up with a firmware with the string "walledgarden" in the name, and all your connections get redirected to Comcast's activation page. The redirection is only DNS-level, though; switch to Google Public DNS servers on your router/DHCP server/computer and you can surf just fine.
  • Speed seems to be enforced per-account, not per-link. That means that they can't apply any speed package to you until you go through the activation wizard, which registers your modem's HFC-side MAC address with Comcast. The "walledgarden" speed actually appears to be Comcast's Ultra package (22/5mbps). That means that if you're not signed up for Ultra or Extreme (50/10), but chose Performance (12/2) or Blast! (16/2) instead, it's actually in your best interest to not activate.
  • If you have to activate, the technician workflow is way easier: no installing software that messes with your network settings, and it works in Linux.
More on the great wireless network project coming soon.

Friday, March 26, 2010

Fun with map visualizations

I've been playing with Google's Chart Tools quite a bit lately.

I did some internal stuff at work (that unfortunately I can't show here) to visualize problems where you have lots of points, all connected by lines that you know the ideal length of, but where those lengths don't lay out properly in 2 dimensions, and figuring out where to put the points is hard anyway.
The most elegant solution seems to be a force-directed graph: pretend the lines are springs, randomize the initial point positions, and simulate physics iteratively. It's slow, and the algorithm is easy to screw up (don't forget friction, or they just orbit each other), but the results are impressive: consistent layouts that are rotation-agnostic each time you run the algorithm. I ended up rendering the results using the static scatter chart. Sadly, the URL syntax is arcane, and the error feedback is nonexistent.
Today, I got to build a visualization that went public:

We had lots of data from the fiber-to-the-home request-for-information site, and needed a way to visualize it. This is a geo map that uses markers. Unfortunately, the API limits you to 400 points, which wasn't enough, so I (at nmlorg's suggestion) screenshotted the map with 400 points at a time and stitched the results; hacky, but functional. We couldn't have used the direct rendering anyway, as it does one AJAX call to Google Maps to look up each point (we were passing in ZIP codes), so it takes ~10 minutes to render.
We considered static maps (custom icons would've been cool), but the URL length limit makes the effective point limit less than 400, and there's only so much stitching I'll put up with.

Tuesday, March 9, 2010

Karaoke project

Friend and sometimes boss mikeyd had a friend ask him to help her with her entry into a karaoke contest. Mikey suggested that I could try out the video recording on the new camera, so I hauled out some of the old sound recording gear, bought some new mics, grabbed the camera and the tripod and headed to Avalon at Mission Bay's penthouse to film.

We shot 14 takes, of which 3 were usable. Top discard reasons:
  • Video exposure problems
  • Artist goofs
  • Background sound (other people in the room)
The internal mic on the camera is pretty much useless for anything but tourist shots, so I recorded audio on MicroTrack II, one mic into each channel. This gave pretty good (though not perfect) separation of the guitar and vocal tracks. When I got home, over several days, I cut the audio into multitrack mode with Soundtrack Pro, adding more vocal to the mix, a bit of stereo separation, fade in and fade out, and a bit of reverb to the vocals. I tried a some light auto-tune, but the guitar strumming picked up on the vocal track sounded really, really odd through the auto-tuner.
I used iMovie (I didn't have time for the Final Cut Pro learning curve) to cut the original audio off, sync the new audio track, trim the junk out of the video, normalize the audio volume, and export 720p to YouTube. Here are the 3 results:






Lessons:
  • Shooting in front of a window? Maybe it would look nicer as a night shot.
  • Leave plenty of time on either end of both sound and video; trying to sneak the last second out of the clip is lame.
  • Get a board to show take number to the camera. Say it clearly on the audio track. Stop and start between each take, even if they screw up the first note.
  • Think you've got 2 or 3 good takes out of 14? You probably need to shoot 30.
  • Put whoever's providing rhythm on a click track, even if you don't think you'll cut multiple videos together. Write the BPM on the cue board.

Sunday, February 28, 2010

Photo of the day

Finished jump 16 today. I'm finally landing the 220 (square foot) parachute without getting dragged along the ground, but I'm still not practicing what I should be in free fall (the last attempts at back flips have ended with me stuck on my back, fighting to get turned over again). After 3 jumps, I stopped and photographed the next load. Favorite picture:



Second (click and look at it in high res -- I love the grass lighting):



Pond swooping is supposed to look like this:


Not like this:


The whole series is here. Video of plane takeoff here.

Lessons:
  • The EyeFi card is slow (class 4 or less), making burst mode painful once I run out of buffer, especially shooting fast-paced stuff like parachute landings. Pro X2 may be better (and supports RAW), but not released yet.
  • No aperture or shutter speed changes in burst mode. Start shooting near sun, end shooting away, end pictures are really dark.
  • Auto-focus has trouble with crazy focus changes while shooting video. Try telling it what object to focus on first?
  • Pictures are way prettier on the big LED-backlit iMac screen at full size than at 400px on this blog post.