First Five (and a Half) Minutes on a Server with a Shell Script

About a year ago I wrote this about hardening a fresh server using Ansible. This post has received about 10x as much traffic as anything else I've written. Oddly, admin is the area I'm probably least knowledgable about when it comes to software.

Anyway, the Ansible script I wrote about in that post is out of date. I realized this recently after trying to use it on a fresh install. I went about updating it (Ansible has made some breaking changes since then) and came to the realization that it would be faster and easier to just write a shell script. 

That's not to say Ansible (and tools like it) don't have their place. They obviously do. But for one-off installs or occasional use the learning curve is too steep. It's much easier to stay current with shell scripting than it is to stay current with a tool that is constantly being changed (improved) and meant for administering very large installations. 

Read more »

A Simple ToDo App in Swift

I decided the best way to learn Swift would be to whip together a simple Core Data app with a single view controller. That gives us a project with some depth but doesn't waste too much time on breadth by minimizing the time spent in IB and wiring up UI components. A ToDo app seemed like an obvious choice.

Our project requirements:

  1. Persist todos (Core Data)
  2. Have a completed state for todos, be able to delete them
  3. Have custom swipe controls for marking complete, deleting
  4. Edit todos in-place and touch to edit
  5. Color todos by position in list (urgency)

This project started with the navigation controller template in Xcode. I won't go over all of the changes or inclusions but you can get a complete copy here if you're interested. I also won't go through the building of the app step-by-step instead I'll just cover some of the more interesting parts with regards to transitioning to Swift.

Read more »

Deploying Yesod to Digital Ocean (Less Painfully)

I recently wrote about deploying Yesod to Ubuntu 14.04 x64 on Digital Ocean here. Since GHC hasn't been updated to 7.8 in the haskell-platform package for Ubuntu, I installed a pre-compiled set of binaries as suggested here. That's not without issues, including symlinking a library (not a great idea) and guessing which packages need to be installed. This is necessary because the binaries were compiled for an earlier version of the OS. In short, it's a horrible way to do it.

The other suggestion on that page is to install the platform from source. After some digging it turns out that this is much, much worse. I gave up after 5 minutes and I doubt anyone familiar with the subject would blame me.

Some more digging revealed that it's vastly more straightforward to build GHC and cabal-install individually. There is no drawback to doing this in practice and building Yesod and Keter works exactly the same. I've put together a shell script for it, lifted mostly from this.

Deploying Yesod to Digital Ocean

[Update] I've outlined a much cleaner way of updating GHC to 7.8 here. This doesn't involve installing the pre-compiled binaries from Ubuntu 12.

I've been working with Haskell about as much as I've been writing on this blog -- very little. So, two birds...

I've got a couple projects coming up that need websites built so I decided that would be a good excuse to learn some more Haskell. After digging around I decided Yesod is likely the most mature web framework so I tried it out. I didn't want to stop at the "install and run locally" phase as that gives a false sense of simplicity when dealing with web frameworks. I think it's more telling to set up a demo site and then deploy it to a (near) production setting. 

I host everything on Digital Ocean these days. I wrote previously about a quick way to harden a virgin install to make it more secure and easier to use. I didn't bother this time since everything was torn down quickly but if you're looking at a production install you'll want to think about making it more secure.

The preferred way of running Yesod in production is Keter, which acts as a reverse proxy, monitors applications, gracefully deploys new versions and some other fun stuff. The Keter docs cover everything very well. What is complicated about it is the fact that we're going to need to compile a binary version of Keter and it's not a good idea to compile that binary on your production machine. The same goes for the Yesod binaries. Both the Keter and Yesod docs talk about why, so I won't do it here.

Why is this a problem? Well unless you're running the same OS locally and on your server you're going to have to put together another server or a virtual machine that you can compile the binaries on.  Given that my SSD rMBP is woefully light in the storage department I opted for the former. Why not just compile on the same machine? Well, for just trying things out we could do that, but the whole point of testing this is to see what the Haskell ecosystem is like for actually getting the site up and running.

What I ended up doing is firing up two instances (droplets) of Ubuntu 14.04 x64, one a minimal $5/month version that will hold the production site and another, beefier, $40/month droplet (dual core, 4GB ram) to compile our binaries on. Don't worry about the cost as you'll probably be using it for less than an hour and will only be billed for that much.

The main issue you're going to run into here is the available Haskell platform package's GHC version is 7.6 while the newer version is 7.8 (which is what I installed on my Mac). As far as I can tell, projects created with Yesod for 7.6 are not compatible with 7.8. In order to work around this we can either install a binary bundle of the Haskell platform or compile from source (available on the same page). I thought the latter was more of a gamble so I decided to use the pre-compiled binaries. The page does say that it works on Ubuntu 14. Of course, it then trails off with something, something, extra packages, something, symlink. 

Good news, I figured out what all of that extra stuff is so you don't have to. The short version is that the Ubuntu 14 image (at least the one DO is using) is missing a few things, but that's easy enough to sort out.

Read more »

[Ab]using Blocks for Cleaner MVC in Obj-C

As I've started to utilize blocks more in iOS/OS X development I've noticed a patter emerge and wanted to talk about it. It's using the same building blocks (excuse the pun) are you're likely to find in any Cocoa project but leveraging blocks to the fullest extent has sped up development time for me and led to both thin controllers (which I think are good) and a very strict separation between the different layers in MVC. 

(Note: I wanted to point out that MVC in Cocoa in general, and explicitly in the example I give here, is more accurately called Model-View-Adapter as the model and view layers do not interact directly with one another, as they would be allowed to do in traditional MVC.)

I won't talk about blocks since they've been around for a while but if you're not familiar with Obj-C a block is an anonymous / first class function. It grants Obj-C (and C and C++) a handy tool for adopting a more functional programming style. The syntax can be awkward at first and I recommend this site as a handy reference and for an occasional laugh. 

So what does a block have to do with MVC? In MVC the controller layer is responsible for mediating between the model and view layers and usually "owns" or manages the lifecycle of both. While in theory a controller will generally be "thin", doing no more than it has to do to tie model to view, they tend to bloat over time. 

In very practical terms, controllers usually have a lot of functions defined in them. For every possible action in a view layer the controller will usually have a separate function (or a shared function with additional logic to determine the sender and action required). Working with Xcode and IB, that means defining and implementing a function as well as making a connection for that action in IB. Since we usually need a reference to the sender(s) (think a set of buttons with mutually exclusive state) we also end up defining properties. That's a lot of "stuff" for, say, an "Open File" button.

Read more »

Converting MIDI Pulses Per Quarter (PPQ) to Samples

I recently wrapped up an audio project for OS X based on JUCE (a cross platform C++ library for applications with an emphasis on audio) and using Propellerhead's ReWire protocol to allow the application to feed audio with very low latency to another audio app (e.g., Logic Pro).

The ReWire protocol defines a host, a device and a GUI (or "panel") that controls the device. The ReWire ecosystem is divided up that way so a ReWire device can be run inside a host (in the host's address space) in a similar way to how Audio Units or VSTs run as "plugins" in a host application. In order to control the device, a GUI or panel communicates with it via inter-process communication. (For standard plugins, this GUI would be directly managed by the host application.)

ReWire speaks both MIDI and audio data. It also communicates transport position and control commands (play, stop, etc) in both directions, allowing the transports of two applications to run in sync. For transport position, it uses MIDI PPQ (also called ticks or just PPQ) to give an offset. If you're building an audio data only application, as I was in this case, your ReWire device will probably ignore the MIDI side of ReWire and therefore will have no reference to utilize PPQ with. That means you'll need to convert a PPQ offset to a sample offset.

Even if you're ignoring the MIDI side of ReWire, the host will still announce all of the information you need to determine the current sample offset of the transport.

A few definitions first:

  • PPQ (pulses per quarter note) is expressed as its smallest subdivision, so PPQ 96 means a maximum resolution of 1/96th of a quarter note. ReWire uses PPQ 15360.
  • BPM (beats per minute) as our tempo. In this case we can assume each beat is a quarter note. The actual value ReWire supplies is BPM * 1000.
  • Sample rate is the number of samples per second (per channel) of our audio. In this case we need to use the sample rate as announced by the host, not by any of the audio files we're playing back (though we need to convert the sample rate of each audio file to that of the host for playback, so they'll end up being equivalent).

To convert PPQ to samples, we'll first convert our PPQ offset to milliseconds. The formula for that is milliseconds = milliseconds per minute / (BPM * PPQ) or in this case 60000 / ((120000 / 1000) * 15360) or 0.03255208333 ms per tick for a tempo of 120 BPM. Our offset in milliseconds is then PPQ offset * milliseconds per tick. To convert that value to a sample offset, we multiply by our sample rate (divided by 1000) or sample offset = (PPQ offset * milliseconds per tick) * (sample rate / 1000).

Putting it all together, if we have a tempo of 120 BPM, a sample rate of 44100, a PPQ value of 15360 and a PPQ offset of 1 million, we'll end up with ((60000 / ((120000 / 1000) * 15360)) * 1000000) * (44100 / 1000) or approximately 1435546 samples.

Since our PPQ value won't change (it's always 15360 for ReWire) we can simplify this quite a bit. Here is a simplified version in function form (as well as the complementary conversion from samples to PPQ):

First Five (and a Half) Minutes on a Server with Ansible

Note: The Ansible script below is unusable due to breaking changes. I've written about a similar approach here using a simple shell script.

This is a response/addendum to two really good "first five minutes" style posts discussing the setting up and basic hardening of a remote server. Brian Kennedy discusses his first five minutes here1 on Ubuntu. It's a great tutorial covering the basics of security. Of course, if you've gone through it once you'll want to automate it. There is also a post on automating the process2 (actually using the steps described in Brian's post) with Ansible. The latter was either not tested or only worked on earlier version of Ubuntu/Ansible. I'll cover an updated version here that works with the most recent version of Ansible and Ubuntu 13.04 x64 and includes some helpful additions. 

So, starting from a virgin install of Ubuntu server we're going to want to perform the following steps:

  1. Update & upgrade the system via apt-get
  2. Install vim & mosh (personal preferences)
  3. Install fail2ban to block ssh brute-force attempts
  4. Reset our root password to something strong
  5. Create a new user so we don't have to use root
  6. Copy over our pub key
  7. Lock down sudo
  8. Lock down ssh to prevent root & password login
  9. Setup the ufw firewall
  10. Configure unattended security upgrades
  11. Configure logwatch to email daily server logs

Even if you can do all of that in five minutes, this is obviously complicated enough that we want an automation tool to handle it. After reviewing popular automation tools like Chef and Puppet, I decided to go with the slightly lesser known and arguably simpler Ansible. Ansible is simpler because it doesn't require any server side installs to work. All Ansible commands are run via ssh from your computer and only need a password or private key to run. Ansible commands are organized in "playbooks" and Ansible has a extensive set of modules that simplify common tasks.

Read more »

Building a Fast Web App in 2013

I recently decided to put some other projects aside and build a web app/service that's sort of a "scratch my own itch project". My aspirations for it go beyond a personal project, however, so I wanted to build it in a way it could handle a lot of users without too much maintenance and resources. 

So what language/framework to choose? PHP is still the most popular web language (according to a recent analysis1 of jobs posted on Twitter) but, well, it's PHP. Rails (and to a lesser extent Django) is very popular, though it's a "kitchen sink" framework and doesn't have the best reputation for performance. Node.js is interesting, not because I care that much about using one language for the server and the client (especially if that language is JavaScript), but because it's supposed to be fast. 

Since i started playing around with Haskell recently I also took a look at the most popular web frameworks: Yesod, Happstack and Snap. Not only are there three somewhat mature web frameworks (with documentation that ranges from ok to pretty good) there's some evidence that Haskell web apps can be fast, maybe even really fast2. Despite the promising start, I ended up dropping Haskell.

Read more »

LJSelectionView Now Available on GitHub

I recently pushed LJSelectionView to GitHub. The project makes it easy to manage an NSView with a collection of subviews and their selection -- either by mouse clicks or "drag to select" actions. Selection rectangles, highlighting and selection management is something I've had to write more than once for Cocoa apps so I decided to write a standalone version to share.

I've set this up as a controller with several NSView subclasses. The controller is responsible for managing selection (and undo/redo) and the views are fairly dumb affairs that mostly just draw themselves (just how it should be). The exception is the main LJSelectionView that also understands the difference between selection, highlighting and your "content" views and has methods for managing them.

The two other views, LJSelectionItemView and LJSelectionRectView are configurable (up to a point) for line color, line width, fill, etc to change what the selection and highlighting rectangles (the stuff drawn around selected views)  look like. If you want to go crazy with their appearance they're easy enough to modify or subclass.

For the ARC-shy (not that there aren't good reasons to avoid ARC at this point) all files will support ARC or non-ARC project automatically. It cluttered up the code a bit but I thought it was better than supporting one vs the other.

The selection behavior is the same as Adobe Illustrator: If you hold the shift key while clicking or dragging, you toggle whatever is being selected against the current selection. If you're not holding the shift key, whatever is selected always replaces the current selection. As it is now, views aren't selected until after the selection operation is completed. It shouldn't be that hard to change it to support "live" selection but again I'm following Illustrator here.

To use, just copy the four main classes and their headers (what's in the root of the GitHub repo) to your project. You'll also want to setup the view hierarchy in IB and make sure the correct outlets are connected. It is possible to setup the view hierarchy without IB you just have to make sure the right connections are made. You can look at the demo project to see how it's setup. The tests aren't automated in any way but you can run them in Xcode with ⌘-U as normal.

The whole thing is MIT licensed so do with it what you will.

My First Algorithm in Haskell - Steps 2-5

This is a continuation of a series of posts starting here and following this. These three posts were front loaded since most of the work (and interesting explanations) were in the foundational work for the algorithm, with the actual steps (the steps as described in the original paper) being mostly data and application of the functions we wrote in part one.

Read more »