Mobile game development: putting finishing touches on visuals and sound

For my latest mobile game project, I decided to spent more effort on the overall player experience, in particular the visuals and sound. After I worked out the basic gameplay rules and level design, before releasing the game I went back to put some of these finishing touches on.

For the experienced mobile game developer, these types of things are second nature, but for the hobby game developer who is still learning the ropes it can take some time to shift one’s thinking to be player-centric. These refinements also contribute to the game’s overall appearance and can result in more downloads if they are showcased in screenshots and app preview videos. Ultimately, one of the best types of marketing is a game that markets itself with appealing graphics.

In this post I’ll highlight some of the visual improvements I did, followed by some audio and sound ones.

Visual

  1. I used a star field animation rather than a plain background on the screen behind the game board. This took some tweaking to get right, and I ended up using different layers of stars to add depth, with colors and speeds set accordingly. For example, stars that are closer move faster and are brighter looking. The stars also fit well with the ‘meteor’ element which is introduced into the later levels.
  2. I added an animation at the beginning at each level where the tiles fall into place from the left and right sides of the screen.
  3. When a game ends, the tiles fall away off the screen.
  4. Touching and holding the screen causes the tiles in the same column and row to be highlighted up. This has to be done since the actual square being clicked on is usually not visible because the user’s finger covers it. The color of the highlighting is white for valid spaces, and red for invalid ones.
  5. I spent a great deal of time tweaking the animation for when tiles expand or ‘grow’ outwards. I ended up with using a random timing for when tiles appear, and this gives the game a certain chaotic feel I thought was fitting. Also, rather than a tile suddenly appearing, the color grows quickly from the center of the tile.
  6. I added a white square showing the last tile placed by each player, since it’s easy to forget.

Sound/Audio

  1. I added background music which had a retro feel to match with the visuals, and a high-energy feel to make the game more exciting.
  2. I added sounds corresponding to winning or loosing a game.
  3. I added a sound signifying that the user tried to place a tile at an illegal location.
  4. I added a sound for when the tiles fall into place at the beginning of each level.
  5. For each tile appearing as part of the expansion process, I play a chime-like sound that gradually increases pitch.

For the app preview video, I found another high-energy song which I really liked so I used that instead of the game’s actual music. I also shut off the sound effects since I thought they would be distracting. But the visuals are all intact, so if you are interested you can check out the app preview video here.

Lessons learned doing sound on iOS – the real story

A few days ago I wrote an article about using sound effects in iOS games. Just today I was doing some more testing and realized I had made a major mistake, and some things I said in that post were wrong. In this post I’ll let you know what I had did wrong.

I alluded to a single line change that would allow playback of MP3 files. Here is the specific line:

NSString *audioFilePath = [[NSBundle mainBundle] pathForResource:sampleName ofType:@”caf”];

I had simply changed the “caf” to “mp3” and since I was getting sounds I assumed things were working. The weird staticky sounds I got I explained by saying that several of the same sound were colliding and causing such an effect. Though sound collision can be a problem and sometimes it might make sense to have several variations of a sound effect, what appears to have been actually happening is that OpenAL didn’t know how to process MP3 and so it was just generating random noise.

After a tried a few different .MP3 files and they all sounded pretty much the same, I finally realized what was going on.

The solution was to return the code to use “caf” files, and use a line like the below to convert MP3 files to CAF files.

> afconvert -f caff -d LEI16@44100 chop.mp3 chop.caf

After doing this, my sounds finally started playing properly. I did find some strange effects that occur when sounds overlap quickly, but they were more subtle.

One additional step that is good to know is you will need to manually tell the .caf files you added to your project that they need to be included with the bundle that gets copied to the device. You can do this by dragging the file under XCode’s Build Phases => Copy Bundle Resources. I didn’t notice this problem for MP3 files, as it seems smart enough to add the to this list automatically.

The strange thing is I would expect OpenAL to throw an error somewhere instead of just spitting out noise. So either this set of APIs is pretty badly designed, or there is some code in AudioSamplePlayer that isn’t handling things properly. Either way, I got things working so not going to pry any deeper at this point.

Lessons learned doing game sound in iOS

[Note: there are some problem with the conclusions drawn in this post. See this follow up post for corrections]

For my current mobile game project, I have a need to do both background music, and sound effects.

In a previous project, I used AVAudioPlayer to loop a MP3 file continuously in the background, and you can see an example of code to do that here (just make sure you call setNumberOfLoops to “-1” if you want infinite looping). For the sound effects, originally I planned to use the same API, however there are some problems with this. The biggest of these is there can be a large delay (a second or more) between when you kick off playback and the sound actually plays. This is fine for background music, but for a series of quick sound effects (imagine the sound of bricks falling, like in Tetris) it just doesn’t fly.

One of the other ways to play sound files with a bit more control is OpenAL. The documentation by apple (here) is pretty good, but it has a pretty steep learning curve, especially if you are like me and didn’t want to spend hours on experimenting. One of the reasons for this is if you want to do multiple sounds you have to do extra work to manage the simultaneous sounds, since unfortunately OpenAL doesn’t do this out of the box.

Fortunately, someone put together a class called AudioSamplePlayer (get it here) which covers much of the basics needed for playing simultaneous sounds, plus the added bonus of easily modifying the pitch and volume of them. The class is pretty easy to use, with the only minor annoyance that you need to pre-load sounds. However, this is understandable since it is designed to improve performance by avoiding loading each sample every time it is played. The licensing of the code is also such that you can use it pretty freely (but be sure to read it in full before using it yourself).

With AudioSamplePlayer you can save some development time, but you may need to do extra work depending on your requirements. One minor tweak I made is to allow .mp3 files (instead of just .caf files) which was a simple one line change.

A bigger problem is that if you game has the possibility to play the same sound quickly in repetition, the sounds can overlap and the resultant sound can sound strange (tinny or mechanical). For example, if you were playing the sound of each rocks hitting the ground in a simulation with 1000 rocks, some would likely occur near the same time.

In this case, one option is to limit the number of simultaneous sounds (and the above class allows this with a simple constant), but that can lead to unexpected behavior as well, especially if your sound file is relatively long (this includes and echo or space at the end of the file). You can also write logic to use a few different sounds to avoid exact-file collisions, or modulate the sounds in some way. You could even delay sounds a bit to reduce collisions, but again that would often be undesirable since the visuals would be out of sync with the sound.

In my case, I am considering just allowing the sound collisions since the resultant effect sounds sort of cool and seems to fit the atmosphere of the game.

Your next question is probably where to get the sound effects, since you may not want to record them yourself. There is a bunch of free sound effects sites on there, but I’ve found this site to have many interesting effects which can be used given the condition that you attribute the site.

References

http://www.raywenderlich.com/69369/audio-tutorial-ios-playing-audio-programatically-2014-edition

https://developer.apple.com/library/ios/documentation/AudioVideo/Conceptual/MultimediaPG/UsingAudio/UsingAudio.html

http://ohno789.blogspot.com/2013/08/openal-on-ios.html

http://www.freesfx.co.uk