Willow Tree Apps

5 Factors in Mobile App Localization

ron.duplain

Nous avons créé cette appli. We built that app, in as many as 16 locales in a single app install. I’d like to share our experience, to help understand the costs and requirements in building a mobile application for internationalization (i18n) and localization (L10n).

First, some context. Our mobile applications start with a detailed UX/UI design. These designs include pixel-perfect layouts and assets, copy for articles and documents, and microcopy for the various dialogs and buttons throughout the application. A successful localization process must take the entire design into account — it’s not just a matter of translating text.

Ready for an international audience? Here are factors to consider:

1. You need translators. That’s a given, but a good workflow in managing content across your target locales is essential! You can save on costs by passing around Excel spreadsheets between content owners, but this kind of unstructured process will incur costs later on. Mistakes in the spreadsheet will slow down development, and copy/paste in Microsoft Office can sometimes lead to inconsistent or unusual character encodings, which cause infamous unicode errors.

A good workflow will support the quality control process, the next two factors:

2. Translations do not always fit within the initial design context. A button which says “OK” in English might prefer a different dialog workflow in another locale. Counting “2 out of 7 total” is reversed in some locales, translating roughly to “7 total, currently 2” where the numbers are reversed.

3. Translations do not always fit within the initial design layout. Translating copy could kill the pixel-perfect layouts in the design, where a one-liner in English could be four lines in German. This is especially challenging when image assets include words which must be changed in another locale, which can force the translated image to change dimensions.

4. Accept the OS’s localization setting. Do not circumvent it. You might be tempted to add a language/locale selector to your app, and you might find a developer willing to implement it. Don’t. Both iOS and Android are doing an impressive amount of work to choose the right text and assets when displaying your app. Any attempt to build this into the application itself will not only be a lot of work to build in the first place, it will work against the OS, break unexpectedly, and cause a headache for everyone involved (and the OS developers will not have sympathy).

This means you’ll need devices on hand which have the OS configured to allow selection of your target locales.

5. Note the increased app size of localization. If you deploy one application across all target locales, it will include the text and assets of each locale, no matter which one is used. This is by design; it lets the OS figure it out depending on the user’s system wide setting. If your user only uses one locale, they have all of the other supported locales sitting around taking up space, with a larger initial download.  This is especially important if your application requires translation of its images.  Note it’s much simpler to publish one app with all locales, than to try to distribute your application to each locale individually.

When you publish your application internationally, note export controls on encryption software, including HTTPS (which many apps have).

Best mobile app development company in the world!

Tobias Dengel

SourcingLine, and independent research company, just ranked WillowTree Apps as its number one mobile development company among its list of top app design and development companies in the world.  SourcingLine gave us best rankings for our profile, portfolio and references across iPhone, Android, Windows Phone and other platforms.  Check out the the SourcingLine top rankings of iPhone, Android and other mobile app development companies here:  http://www.sourcingline.com/research/top-mobile-application-developers

Apps for Venues goes live with Sprint Center

Tobias Dengel

TV station covers new Valpak Apps

Tobias Dengel

WPIX did a great story on the new Valpak apps, and couponing in general. (WillowTree helped design and develop the Valpak iPhone, iPad and Android apps)…..

Sitter Sorter included in “Date Night Essentials” on iTunes Store!

Roger Casey

We got a wonderful surprise on Friday when we discovered that the Sitter Sorter app is now being featured under the “Date Night Essentials” category in the iTunes store, sharing limited space with Yelp, Netflix, and other major players’ apps.  Congratulations to our friends at Sitter Sorter!

Sitter Sorter, an iPhone application developed by WillowTree Apps, designed to make finding and organizing your babysitting needs easier, is a must for any parent who wants to be able to find and manage their babysitters with ease.

Download it from the iTunes store here.

CNET covers the Likes! Directory App Debut

Tobias Dengel

CNET’s Rafe Needleman covers the launch of WillowTree’s Likes! app, and compares to other Facebook apps. Likes! is available on iPhone, Android and as a Facebook App.

Macworld Covers Likes! Directory App Debut

Tobias Dengel

Macworld’s Alexandra Chang did a complete review of WillowTree’s Likes! app, which is designed to turn your Facebook friends’ likes into a directory and recommendation engine. Click here to download the Likes! app for iPhone, Android or Facebook.

WillowTree Apps Launches Likes! Facebook App

Tobias Dengel

Check out the Likes! app for iPhone, Android and also available as a mobile and desktop Facebook app.   Press release came out today.  The app permits users to organize the billions of Facebook likes into a directory/recommendation engine based on what their friends (and friends of friends) like.

Some Musings on AVFoundation in iOS

pomalley

One common dilemma in programming is that the simplest tools to use for a given task are often the least flexible. Higher degrees of versatility and customization almost invariably lead to a higher learning curve for the developer. Before Apple made AVFoundation available for iOS, there were no intermediary steps between the relatively simple, but restrictive, UIImagePicker and the very powerful but exponentially more confusing Core Video/Audio tools. We had the chance to get into the inner workings of AVFoundation in one of our recent projects and, although it really does open up a lot of functionality that was previously arcane and inaccessible, we discovered a fair amount of quirks that took a fair amount of trial and error to work around.

For those who are unfamiliar with how AVFoundation is set up, the first step is specifying the relevant inputs, the device cameras or microphone, and outputs, which can be already optimized for photo/movie file generation or the raw video or audio data. These are then added to an AVCaptureSession and from that point on AVFoundation takes care of any necessary coordination between them. This part isn’t too difficult to get rolling, but where we starting running into trouble is identifying what can and can’t be done using the various tools at our disposal. For our project, we needed to be able to apply various transforms to flip, mirror, and crop the incoming video data. Flipping and mirroring are easily accommodated with the preconfigured movie file output by finding the AVCaptureConnection between the output and the video input and setting its orientation and mirrored properties. However, there’s no step in this process where we can apply a CGAffineTransform or something similar to scale the incoming images, so cropping was impossible. There is a property called VideoScaleAndCropFactor added in iOS 5.0 that looked promising, but as we learned when we tried to use it, it only works with photos. In the case of video connections, both the minimum and maximum value for this property is set to 1, so it wasn’t very useful for what we were trying to do.

Our next attempt at getting the desired behavior was by using an AVAssetExportSession to process the captured video and apply the appropriate transform to the finished video before writing to the library. While this worked, it led to processing times that were orders of magnitude larger than we were seeing before. It usually only takes about one second for each minute of recorded video to write to the library, but because now we were going back and applying transforms to every single frame of the video before even starting to write to the library, the whole ordeal took 50-100% longer than the time actually spent recording. This wasn’t going to work for our app, so we were back to square one. We were beginning to think that what we wanted to do was simply not possible at this point, but after scouring the web for ideas and enough failed attempts to fill my photo roll with garbled videos, we managed to get an approach working using the basic video and audio data and AVAssetWriter that applies a transform to all of the incoming video data, rips the image out and crops it, creates a pixel buffer for the new image, and appends that buffer to the movie file instead of the original. The initial transform is created by applying a CGAffineTransform to the AVAssetWriter’s video input at initialization, which scales up the initial images (by 65% in this case) so that they’re ready for cropping, as well as rotating them to the appropriate orientation based on whether we’re in portrait or landscape mode. The really tricky part, though, was doing the image manipulation and pixel buffer generation as the frames are coming in. The code for this is all in the delegate method used to handle both the incoming video and audio data (see below). This code does make use of some instance variables, but most of it is self-contained.

- (void)captureOutput:(AVCaptureOutput *)captureOutput
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer
fromConnection:(AVCaptureConnection *)connection
{
if( !CMSampleBufferDataIsReady(sampleBuffer) )
{
NSLog( @”sample buffer is not ready. Skipping sample” );
return;
}

if( _cameraBar.recording == YES)
{
_lastTimestamp = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);
if( _videoWriter.status != AVAssetWriterStatusWriting) { //initialize the video at the first frame
[_videoWriter startWriting];
[_videoWriter startSessionAtSourceTime:_lastTimestamp];
_startTime = CMTimeGetSeconds(_lastTimestamp);
} else if( captureOutput == _videoDataOutput) {            //skipping the first frame to ensure the video writer is ready
CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
// Lock the base address of the pixel buffer.
CVPixelBufferLockBaseAddress(imageBuffer,0);
// Get the number of bytes per row for the pixel buffer.
size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
// Get the pixel buffer width and height.
size_t width = CVPixelBufferGetWidth(imageBuffer);
size_t height = CVPixelBufferGetHeight(imageBuffer);

// Get the base address of the pixel buffer.
void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer);

// Get the data size for pixel buffer planes
size_t bufferSize = CVPixelBufferGetDataSize(imageBuffer);

// Create a Quartz data provider that uses the data
CGDataProviderRef dataProvider = CGDataProviderCreateWithData(NULL, baseAddress, bufferSize, NULL);

// Create a bitmap image from the data
CGImageRef cgImage =
CGImageCreate(width, height, 8, 32, bytesPerRow,
_colorSpace, kCGImageAlphaNoneSkipFirst |
kCGBitmapByteOrder32Little,
dataProvider, NULL, true, kCGRenderingIntentDefault);
CGDataProviderRelease(dataProvider);

CVPixelBufferRef croppedBuffer = NULL;

NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey,
[NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey, nil];

CGImageRef croppedImage;
CGContextRef croppedContext;
size_t croppedWidth = width / 1.65;
size_t croppedHeight = height / 1.65;

//Crop the image to simulate a 65% zoom
croppedImage = CGImageCreateWithImageInRect(cgImage, CGRectMake(width * (.65/1.65/2), height * (.65 / 1.65/2), croppedWidth, croppedHeight));

//Create a new pixel buffer and draw the cropped image into its context
CVPixelBufferCreate(kCFAllocatorDefault, croppedWidth, croppedHeight, CVPixelBufferGetPixelFormatType(imageBuffer), (CFDictionaryRef) options, &croppedBuffer);

CVPixelBufferLockBaseAddress(croppedBuffer,0);
void *croppedAddress = CVPixelBufferGetBaseAddress(croppedBuffer);

croppedContext = CGBitmapContextCreate(croppedAddress, croppedWidth, croppedHeight, 8, CVPixelBufferGetBytesPerRow(croppedBuffer), _colorSpace, kCGImageAlphaNoneSkipFirst | kCGBitmapByteOrder32Little);

CGContextDrawImage(croppedContext, CGRectMake(0, 0, croppedWidth, croppedHeight), croppedImage);

CGImageRelease(cgImage);
CGImageRelease(croppedImage);
CVPixelBufferUnlockBaseAddress(imageBuffer, 0);
CVPixelBufferUnlockBaseAddress(croppedBuffer, 0);
CGContextRelease(croppedContext);

//Append the new buffer into our video
if (croppedBuffer) [self newVideoSampleWithBuffer:croppedBuffer time:_lastTimestamp];

} else if (captureOutput == _audioDataOutput) {         //Audio is a lot simpler, thankfully
[self newAudioSample:sampleBuffer];
}
}
}

Note that the actual numbers here could be changed to match any zoom level that you want as long as the resolution that you specify in your session preset and the output settings of your asset writer (which should be equal, by the way) is high enough. We decided to use AVCaptureSessionPreset640X480 to keep file size low, but there are a many higher resolution options to choose from. The methods called to append the data to the file have some validation in there to ensure that we’re not working with junk, but the key lines of code to append the data are here:

[_videoWriterInputPixelAdaptor appendPixelBuffer:croppedBuffer withPresentationTime:_lastTimestamp]
CVPixelBufferRelease(croppedBuffer);

[_audioWriterInput appendSampleBuffer:sampleBuffer]

There are some important things to note here. First and foremost,  doing all this image on the fly takes a lot of processing power, enough power that currently only the 4S can actually create usable video with it. Using this method with the iPhone 4 causes a drop to ~15 fps and none of the audio gets captured at all. The 4S will also start to drop audio if you try to get too fancy with the image manipulation on every frame. What we ended up doing was using this method for the 4S and using one of our other, less elegant solutions for the iPhone 4. Since we wanted to be forward compatible with any future devices but couldn’t use something simple like a respondsToSelector: test to determine processing speed , we decided to use the number of processors on the device for this instead of hard coding a check to determine if the phone is a 4S using the model number. The code for this turned out to just be [[NSProcessInfo processInfo] processorCount], which could be useful if anything else comes up in our projects that a single processor can’t handle, but that was surprisingly obscure bit of information that proved difficult to dig up.

Other than that, some other information worth noting is that the ivars that we are calling on here to append the data are AVAssetWriterInputs that we created when initializing the AVAssetWriter at the start of recording. Another thing is that we have to use an AVAssetWriterInputPixelBufferAdaptor as a middleman to append the video data because AVAssetWriterInputs work with CMSampleBuffers and can’t directly handle the CVPixelBuffers that we’re creating. Lastly, failing to release the created CVPixelBuffer will cause a massive memory leak and will crash the app after about 15 seconds, so don’t forget to do that if you do end up doing something similar to this.

So, at the end of the day, it turned out that there’s no ideal solution to this problem, which is strange considering that one would think AVFoundation would easily be able to handle a simple operation like cropping right out of the box. As we were trying to figure out how to deal with this, there was always some confusion about what the capabilities of AVFoundation really were because the documentation can be hazy to the point of being misleading. For instance, that bit about VideoScaleAndCropFactor being useless for an actual video isn’t very intuitive. Another issue is that when you’re not using the stock photo and movie outputs, you have to manually set all of your video and audio settings, which is a problem when you’re like me and don’t know off the top of your head what a good value for an audio encoder bit rate would be. I did find some numbers to use by looking at what worked for other people, but considering how critical it is for these values to be correct, it’d be nice if Apple at least included at least some explanation of what these things are in their documentation, as well as which ones need to be specified by the developer and which can be safely left to a default value. Still, I can’t fault Apple too much, because they’ve let us do things we’d never be able to do with UIImagePicker without needing to deal directly with Core Video/Audio, which would have been a nightmare. If you’re thinking about using AVFoundation for one of your projects, the best advice I can give is to make sure that you have plenty of time to get acquainted with it and don’t expect things to fall into place without some tinkering.

NCCS Pocket Cancer Care App is live….

Tobias Dengel

The Pocket Cancer Care iPhone app by the National Coalition for Cancer Survivorship (NCCS) has gone live, designed and developed by WillowTree Apps.  This app provides invaluable services to cancer patients and loved ones by helping them manage their communications with doctors and other health care professionals.  It also includes a cancer glossary and integration with the iPhone’s calendar to manage appointments and medications.  Download the NCCS Cancer Care app from iTunes here.