Chris

Installing Dropbox on Debian Sid

I had to install the ubiquitous Dropbox on my Debian Sid box tonight, just thought I’d leave a few notes here to help along someone else who was needing it.

As Dropbox does not provide a Debian package, we’ll use their source package. However, we’ll do it in a way that Dropbox will still be installed under the supervision of the package manager and can still be uninstalled, or installed on other systems at will.

1. Start by downloading the Dropbox source (.tar.gz) from this page
2. Install the dependencies using apt:
   
$ aptitude install libnautilus-extension-dev checkinstall

3. Compile the Dropbox source package:
   
$ ./configure
$ make

4. Create and install a new Debian package from the source install using checkinstall
   
# sudo checkinstall make install

   Just accept the default for the questions, unless you’re building a package for distribution in which case specify the applicable options (i.e. requires)
5. Start dropbox:
   Applications -> Internet -> Dropbox
   OR
   $ dropbox -i

Enjoy!

More tips on installing Dropbox from CLI can be found here.

Edit: Raphaël Hertzog commented below to let us know that he has correctly packaged Dropbox for inclusion in the Debian distribution. Here is his post.

Debian Squeeze

After hearing a lot about it, I decided to check out the new Debian GNU/Linux release: 6.0, informally known as Squeeze.

My home desktop/workstation has been running Ubuntu 10.04 for almost a year. Pretty good, very little fuss but I’m never satisfied with anything. So I downloaded the multiarch 32-bit/64-bit DVD image (pretty cool trick, by the way) and wrote it to a spare USB drive.
Continue reading »

Doing Something

Earlier this year, after realising I spent a lot of my time at home watching average TV shows and surfing the net, I started to make an effort to become more of creator than a consumer. I’ve always had ideas of things I’d think were cool or useful, but never actually got around to doing anything about them. Most of them involved tech, most were well within my skill level to accomplish, nothing happened.

So here’s a look at some personal projects that I’ve been thinking about doing but never really got around to…

• Building an Android application for interfacing with your Powershop account
  
  This is what I’ve been doing recently. Learning about Android application development and filling a hole for an application that I, and others, would find useful. The code is 75% done, but there’s lots of tweaks to be done before I’ll be happy with it. Considering releasing this as open source, but not 100% sure yet
• Doing an awesome case mod
  
  Will hopefully have the case here this weekend – It’s going to be awesome… watch this space
• Blog some more
• Build some useful web applications – i.e. an application that allows a household to split bills between flatmates, an application that analyses you’re phone bill and suggests a better plan or carrier to save you money
  
  Both of these have working prototypes, written in Rails. It quickly got complicated and I started running into Rails’ limitations. Would love to get this going and learn some new languages!
• Practice web design
  
  Although I consider myself a competent developer, I find that design and UX is my weak point. I’d love to develop this weakness and grow my skills
• Redesign my blog
  
  Something original and simpler. I find that my blog is far too complicated for what it actually is, so much so I can sometimes find it distracting when I actually do want to write. Also I want to look at re-implementing it with a new platform such as Jekyll or Hyde
• Grow my on-the-side business
  
  As I’ve had a lot of experience with web development and Linux server management, I’d love to help people who need someone to take care of it for them, or possibly even consult.

Also, since this is me, I had to set myself some rules. The most important one is: Don’t start a new project until the one your working on is done. Without that, I’d probably start them all and finish none…

HP Mini 5102

I bought a Netbook!

Why?

A niche had opened up in my technology-filled life. A gap that my aging Macbook Pro is finding itself less and less able to fill as the battery and various hardware begins to fail. A comfort hole that my Nexus One is too small, and my Desktop is too large to fill. A need that the iPad is too limited to fulfill.

What?

An HP Mini 5102 netbook with the following specifications:

• Intel Atom N450 Processor with 2GB of RAM
• 80GB SSD Hard Drive
• Wireless a/b/g/n adapter
• 3G cellular modem
• High resolution 10.1″ touch screen
• 6 Cell battery rated at ~10 hours

Where?

From New Zealand technology outlet, Ascent. Thanks to ibeardslee from #ubuntu-nz on Freenode IRC for putting the wheels in motion to get these in the country!

How (is it working out)?

Just dandy!

The system doesn’t come with any operating system loaded, but the hardware is 99% compatible with Ubuntu 10.04 LTS. The only gotcha being the 3G modem… It apparently requires some sort of firmware to be loaded by a program that only runs on Windows but I haven’t actually tried to get it working because it pairs flawlessly over Bluetooth to my Nexus One for mobile data.

Everything else from the card reader to the camera works, and I’ve been able to get between 8-10hrs battery life out of normal use (wireless surfing).

Any tips?

• I found I had to adjust the DPI for the screen as it’s very high resolution (1366×768 on a 10.1″ screen) so my eyes aren’t torn from their sockets. I found that around 120 DPI (up from 96) works quite well. Here’s how you change it:
  System -> Preferences -> Appearance -> Fonts tab: click "Details" and adjust there
• As great as Ubuntu is, it can be a little heavy at times, so I’m currently using Arch Linux on it and everything is much, much faster. I hope to put up a guide on getting Arch going on this thing as it’s a great OS for the tweakers among us!
• Switch out Ubuntu’s version of Firefox for Google Chrome or Chromium, much faster. Arch’s Firefox seems OK, though
• Disable bluetooth and wireless if you’re not using them. Dim the screen for maximum battery life!

Enjoy! Leave a comment if you have any questions!

Broken Blog

If you haven’t noticed (I hadn’t!), my blog has been a bit broken for a couple of weeks.

It’s upgraded to WordPress 3.0 now, but that wasn’t without problems and I almost lost all my drafts. Hooray for backups

Keeping in tradition with fiddling with blogging software more than actual blogging, I was having a look around and noticed Jekyll which looks to be a nifty, yet simple, git-based blogging platform.

Maybe one day

The Web and Advertising

Ars Technica have recently put up a post about advertising on their site. They discuss their business reliance on advertising, the effect of running ad-blockers on their site and some trials of methods they’ve used to combat them.

The site has caused a lot of discussion from all sides. Those with an inflated sense of self-entitlement, those that feel that ad-blocking is stealing, and those that want to find a solution that will please both camps…

I used to run an ad-blocker with a very broad blocklist, because ads annoyed me. If I had thought about it I probably would have said “If you don’t want to give it away from free, take it off the web or don’t complain that I block annoying ads”.

However, the more I did think about it, I came to realise a few things…

• Ad-blockers are a product of a market at work. People were able to reduce the cost of viewing a page or site with very little inconvenience to themselves.
• Most of the complaints about ad-blockers aren’t about blocking their ads, rather blocking all advertising without even giving them a chance, as the ad-blocker came with a very broad blacklist
• Ad-blockers also objectify a market failure. The feedback loop between consumer and producer is broken. With no ad-blockers, if a sites advertising was unacceptable, people would stop visiting. With ad-blockers, it doesn’t work like that. Ars’ anti-adblock techniques were a misguided attempt to fix this: If you don’t like the ads, don’t view the content. If the ads are unacceptable, vote with your feet.

Seemingly, a lot of content providers are happy with “Free” or “Freemium” business models… For instance I have no problem writing on here for everybody to read. Where they draw the line is paying. I imagine that the majority of sites that have advertising is to reach break even point.

Most people don’t seem to oppose the idea of advertising on their sites, as long as they are relevant, unobtrusive and useful. I’ve clicked on plenty of Google search ads, along with the better ad networks such as Fusion, The Deck and even ShopStyle.

In my opinion, the best fix for the situation is two-part:

Content producers should not use brute-force, spammy (think of the vibrating “You’re the 999th visitor, you’ve won…”, Evony and the system error message lookalike) and untargeted ads. They should be running ads that are relevant and unoffensive, and their audience may even find useful. Adsense text ads are a good start.

For this to work, however, consumers that insist on running ad-blockers need to be more selective about who they block, and why. Block-by-default is not going to work.

Ars’ move to detect & deny content to people running ad blockers wasn’t a complete backfire – they didn’t convince people that blocking ads is stealing, but they got their members to think about why ads are run and things you can do to view content and still not have annoying ads, resulting in a what seems to be a few subscription sales and a bunch of AdBlock whitelisting. Goes to show what a little engagement can achieve.

I think the solution to this particular problem hasn’t quite materialised yet, but it sure as hell isn’t increasingly intrusive adverts, or paywalls. If I were to guess, I would say that it would look something like Flattr.

The Nexus One

photo credit: owaief89

Firstly, some background. I’ve been an iPhone 3G user for almost 2 years, getting one almost immediately after they were released here in New Zealand. A great phone, no doubt. The app store was wide-ranging, easy to use with quality work for good prices.

As happy as I was with the iPhone, experiencing the “walled garden” when you wanted to escape wasn’t much fun. I wanted to store files, use non-App store applications, tether with more data-friendly networks – but I couldn’t.

A geek at heart, I kept a close eye on what was going on in the world of Android. For the longest time I remembered the worst of my days using Linux, and the reluctance of having that experience on my phone. That, coupled with the fact that the early Android handsets were woefully underpowered (much like the iPhone was) relegated me to the sideline for some time.

Hearing about all the fun my acquaintances were having with their Magic’s – rooting, flashing, tethering and all the other dirty-sounding verbs, I decided I wanted in to the Android world.

A friend of mine signed a contract with their carrier and got a subsidised HTC magic, which he sold to me. The phone was good, perhaps a little limited hardware-wise, but I loved the heavy Google integration and the apps were pretty good. About this time, my iPhone’s screen died – it must have been a sign! No turning back.

So I got myself a Nexus One. I won’t go over the specs, but will go over the subjective things I’ve come across with the phone.

Screen

The screen looks fantastic. The resolution is very high, photos and video’s look stunning. The touch side of things is also very good – sensitive and responsive. The only thing I noticed is that the colour can sometimes seem off – the more brighter colours apear blown out. Also, the phone is difficult to use in direct sunlight. Both these issues are side effects of the AMOLED screen chosen for the Nexus One.

Hardware

The phone is quick. No delays, no waiting, no keyboard lag. All things that were present with the iPhone and the Magic. Web pages render very quick and the radio used in the phone supports the very best connections available in Vodafone NZ’s network.

Operating System

The phone comes with Android 2.1. Android has travelled a long way since 1.x and is a very functional and usable system. Tight integration with Google, Exchange and Facebook brings your life to the phone. It now has multi-touch support in the browser & maps which should help kill the bugbear a lot of switchers worry about.

App store

The Android market has also grown a lot. There are many apps to choose from and most iPhone apps have an equivalent on the Android Market. Albiet a lot of Android apps aren’t as polished as their iPhone counterparts. Android has a lot of problems to solve for it’s developers – having to develop over dozens of different devices is going to start causing problems, and the time they spend on that is the time that Apple’s using to polish and add features to it’s SDK.

So overall, very impressed with the phone and I feel that I’ll be using it for some time – or at least until the new shiny comes along

Google Alerts For Fun and Profit

I had a request today from a customer who wanted to make a list of products he was waiting for, and to be notified when Mighty Ape had them on the site ready to order. I decided that it was going to be problematic to implement this as a Mighty Ape feature, but I did help him sort out something that works just as well, using free tools available already.

Say you wanted to be notified when Mighty Ape makes the Transformers 2: Revenge Of the Fallen Blu-Ray available, here’s how you could go about doing it.

First, construct a Google query that narrows down what you’re after.

Since you’re interested in a product, make sure you add the keyword “site:mightyape.co.nz/product” to the query. That will restrict Google to search within that directory on our site.

We’re only interested in the Blu-Ray version so add that in there, too as “blu-ray“.

Now add some keywords that describe the product. Try and stay away from being too specific, filler words, numbers and punctuation. Something like “transformers revenge fallen“.

Now, using that might turn up some strange results, from other product’s descriptions or similar. Luckily, Mighty Ape stores the product name and format all in the page title, so we can direct Google to match only that by using the “allintitle:” keyword.

So put that all together and we have:
allintitle:transformers revenge fallen “blu-ray” site:mightyape.co.nz/product

Nice. Now all we need to do is put that into a new Google Alert and Google will let you know when new documents turn up matching your query in your E-Mail inbox, or RSS feed.

Enjoy! This can easily be adapted to other sites or other queries. Anybody got any other cool examples of useful Google Alerts?

100 Years of Change Adversity

Ars.Technica has a fantastic article up titled “100 years of Big Content fearing technology—in its own words“.

The article basically touches over most major technological advances over the last centry, and digs up what “Big Content” had to say about it at the time. Big Content being Music, Film and Print industries.

Such business-busters as the Xerox machine:

“the day may not be far off when no one need purchase books”

The VCR:

We are going to bleed and bleed and hemorrhage, unless this Congress at least protects one industry that is able to retrieve a surplus balance of trade and whose total future depends on its protection from the savagery and the ravages of this machine.

And of course, MP3:

We’ve come full circle here, as this is the inverse of Sousa; a new technology won’t eliminate the amateurs, it will eliminate all the professionals and leave nothing but amateurs.

Of course, this seems all too familiar. The roadblocks and excuses being thrown up today are the same that were spun up all those years ago, those which failed to come to fruition. The same things are being threatened – the death of industries, the loss of jobs and the destruction of dreams.

However, what can be said of all of these advances is that they caused change. They caused a reshuffle of the industry in question, but eventually everything settled down again to get back to what they were doing: making money. In fact, if I remember correctly, most content industries are in boom, some making more money than they ever had (with the exception of newspapers, but I feel they’re the author of their own demise).

So “Big Content” is quite comfortable where it is, thank you very much, and is apparently quite happy to fight tooth and nail against anything that may cause change. The problem is, these industries have a lot of teeth and too many nails. We have the huge lobby groups, the MP/RIAA and the very prominent spokespeople with their voice in the ear of our politicians.

This is why so many people keep an eye on the laws being passed in our names, on behalf of these industries. We need to watch whats going on in the fields of Copyright and Intellectual Property (don’t even get me started about software patents) before we regulate ourselves out of evolution.

Speeding up INSERTs on MySQL

Was working on a problem today that required a lot of bulk inserts to a MySQL table. I was getting about 200 inserts/second on my development system, which is OK considering there was some minor processing going on.

Since some batches can contain well over a million rows, I started working on how to optimise these queries so we can get them in there faster.

Firstly, watch your indexes on the table. Obviously the more indexes you have on the table the more work the DB has to do on INSERT to maintain them.

Second, if possible, ditch your ORM. Instead of building and hydrating objects for each row in the table used direct/prepared queries. Most ORMs worth their salt can handle this. i.e. Propel can give you direct access to the underlying PDO connection to use your prepared statements.

/** @var $con PDO */
$con = Propel::getConnection();

Obviously if you have business logic tied up in your objects, it’s best to use them instead of duplicating code.

Third, lock the table for a batch of inserts. Here’s some sample code

$i = 0;
$con = Propel::getConnection;
$con->beginTransaction();
 
foreach ($rowsToInsert as $row) {
 
  if ($i % 100 == 0) {
    $con->commit();
    $con->beginTransaction();
  }
 
  TablePeer::insertUsingPreparedStatement($row);
  $i++;
}

If your using MyISAM you can lock the table using the LOCK TABLE statement

$con->exec('LOCK TABLES table_name WRITE');
$con->exec('UNLOCK TABLES');

Depending on how important your data is you can change the (100) value in the $i % conditional to higher or lower. The reason this speeds up Inserts so much is that MySQL won’t flush it’s write cache to the disk until the transaction is finished, as opposed to every INSERT statement. However, having unflushed data in your cache is dangerous because it may disappear if something happens to the DB server, or get rolled back if your script carks it. Also, since a transaction/lock table call will stop all other access to the table, if it’s frequently read from those queries will be waiting on locks, so it’s good to refresh them frequently.

Using these three methods I almost tripled the performance of my script which now inserts between 550-600 rows per second. Win!

There’s some more tips over here in the MySQL manual and some of the comments are quite helpful too.