Subversion 1.7 on Ubuntu 12.04

Ubuntu’s default packages ship Subversion 1.6 with the 12.04 distribution. This is annoying if you want to use the newer Subversion 1.7, without upgrading your OS, and especially if you’re dealing with newer working copies:

svn: The path '.' appears to be part of a Subversion 1.7 or greater
working copy. Please upgrade your Subversion client to use this
working copy.

Turns out there’s an official Subversion PPA on Launchpad, so for Ubuntu 12.04 all you have to do is add the following couple lines to your /etc/apt/sources.list file:

deb http://ppa.launchpad.net/svn/ppa/ubuntu precise main 
deb-src http://ppa.launchpad.net/svn/ppa/ubuntu precise main 

And upgrade Subversion to its latest version with apt-get:

sudo apt-get update
sudo apt-get install subversion

At this point you should be running the latest, and the greatest:

$ svn --version
svn, version 1.7.8 (r1419691)
   compiled Mar 23 2013, 23:38:40

Hope this helps!

WordPress Workflow Tips: Using Subversion Externals for Plugins, Themes and Core

Subversion Externals for WordPress Projects

I’m pretty sure most of you are familiar with version control, and that it’s best practice to start each new project in version control, and not “add it sometime later.” Version control will help you manage changes to your code, and have revisions which you can easily roll back to.

Although my favorite version control software is Git, today we’ll talk about Subversion, and WordPress developers know that WordPress itself, as well as the plugins repository and the themes directory, are hosted in public Subversion repos. As you’ll see, this makes it incredibly easy to access core, themes and plugins using external references.

It’s a great idea to keep all of your code under version control, including WordPress core, the themes and plugins that you use. However, themes and plugins, as well as core, are already under version control in a different place, so to avoid redundancy, and extra maintenance work, we can use external references, which are more often referred to as svn:externals in the world of Subversion, or submodules in Git.

In short, an external reference is an entry in your own version control repository, which points to a different place, similar to a symbolic link in your operating system. When running svn update, Subversion will follow these links and grab the source code from those external repositories, without having to keep them under its own version control.

That might sound a little confusing, so let’s take a look at some examples, which will hopefully clear that up.

Plugins as Subversion Externals

Suppose I’m working on a typical WordPress project, and have got some themes and plugins in my Subversion repository, like this:

/public_html/
  - wp-content
    - themes
      - twentyten
      - twentyeleven
    - plugins
      - akismet
      - jetpack
      - bbpress
  - wp-includes
  - wp-admin
  - ...

Note, that the whole public_html directory is under version control.

Let’s say Akismet just shipped a new version, and my WordPress dashboard prompted me to update. When updating the plugin, WordPress will replace the old Akismet plugin files with the new files, thus, my files are now out of sync with my repository. In order to bring them back in sync, I have to SSH onto my server, and commit the new and modified Akismet files back to the repository, which is a pain.

Luckily, we can get rid of such pain by using svn:externals. So lets remove the akismet directory from our repository, and add an external reference to the Akismet plugin, like so:

cd public_html/wp-content/
svn rm plugins/akismet
svn commit -m "Removing Akismet from the repository."
svn propedit svn:externals plugins

Which will launch a text editor (typically nano or vim), where you can add the external reference to the latest version of Akismet from the WordPress.org plugins repository, like this:

akismet http://plugins.svn.wordpress.org/akismet/tags/2.5.6/

Save and quit your text editor, and run svn update. This will update your files from your repository as it usually does, and then update its external references, so you’ll probably see something like this:

Fetching external item into 'akismet'

Followed by the addition of an akismet directory in your plugins folder, with the Akismet plugin source files. These files are not under your version control, they’re just copied over from WordPress.org repository.

Now, when Akismet ships a new version of their plugin, say 2.5.7, you don’t need to update from your WordPress dashboard, you’ll just need to edit your external references, and change that tag to 2.5.7. Then run svn update and Subversion will pull in the new version for you, replacing the old files.

Let’s do the same for the Jetpack and bbPress plugins. Remove them from the repository, and add them as external references instead:

cd public_html/wp-content/
svn rm plugins/bbpress plugins/jetpack
svn commit -m "Removing bbPress and Jetpack from the repository"
svn propedit svn:externals plugins

Akismet is already an external reference, so let’s just add the two new ones:

akismet http://plugins.svn.wordpress.org/akismet/tags/2.5.6/
jetpack http://plugins.svn.wordpress.org/jetpack/tags/1.7/
bbpress http://plugins.svn.wordpress.org/bbpress/tags/2.1.2/

Now run svn update and Subversion will pull in the Jetpack and bbPress plugins for you, directly from the WordPress.org repository. If you’re wondering where I got those URLs, browse to any plugin on WordPress.org and click the Developers tab. You’ll see an svn link to the current version.

Themes as Subversion Externals

You can do the exact same thing with WordPress themes, which are hosted in the WordPress.org repository, such as Twenty Ten and Twenty Eleven:

cd public_html/wp-content
svn rm themes/twentyten themes/twentyeleven
svn commit -m "Removing Twenty Ten and Twenty Eleven from repo."
svn propedit svn:externals themes

Now add the references to Twenty Ten and Twenty Eleven, like so:

twentyten http://themes.svn.wordpress.org/twentyten/1.4/
twentyeleven http://themes.svn.wordpress.org/twentyeleven/1.4/

You’ll find the link to the Subversion repository in the Developers tab of any theme on WordPress.org. Make sure you pick the latest stable version, by comparing the version numbers in the repository, to the version number displayed on the theme page. It is not always the last one, since themes go through a review process, and some versions might not be approved, but still listed in Subversion.

Now, when you run svn update from your wp-content directory, you’ll see that Subversion will pull the external references for your themes and plugins. There! You no longer have to maintain themes and plugins from WordPress.org, in your repository! Just svn update every once in a while ;)

WordPress as a Subversion External

You won’t be surprised that, since WordPress is also hosted in a public Subversion repository, it can be an external too! It’s slightly more tricky than with themes and plugins, but definitely handy. In this particular case, the external reference needs a folder of their own (otherwise you’d have to list all core files as externals), so your working directory has to look somehow like the following:

/public-html/
  - wp
  - wp-content
    - themes
    - plugins

This means that you’ll have to setup WordPress in a subdirectory, and move your wp-content directory outside of WordPress core (though as an external, WordPress will still have one of its own).

Now, let’s remove the wp directory and place it as an external reference to our public_html folder instead:

cd public_html
svn rm wp
svn commit -m "Removing WordPress from repo."
svn propedit svn:externals .

Note the dot at the end of the call to propedit, meaning we want to edit the property for the current directory. Then, in the text editor, let’s add our reference:

wp http://core.svn.wordpress.org/tags/3.4.2/

After saving and quitting your text editor, run svn update from the root of your repository, and you’ll see that Subversion will grab the WordPress core, and your themes and plugins references. Neat!

Whenever you need to upgrade WordPress, you’ll just change the reference to point to the new version, and update again. You don’t need to delete the files every time you update an external reference, Subversion will do it for you. Also, when updating WordPress, don’t forget to visit the admin page, in case there are updates to the WordPress database.

At this point, you should no longer be maintaining WordPress themes, plugins or even WordPress itself in your own Subversion repository. Let WordPress.org do the work for you, while you focus mainly on your own code. I mentioned in an earlier blog post, that you can use GitHub projects as subversion externals too, which is pretty awesome.

If you’d like to learn more about Subversion externals, visit this page.

So what about you? Are you using version control for your WordPress projects? Do you prefer Git or Subversion? Do you currently use, or plan to use external references? Please share your thougts via comments, and thank you so much for stopping by!

Restore A Subversion Repository From Backup Files

This is a quickie and yeh, I don’t like Subversion either, and if you’re still using that consider Git or Mercurial. Anyways, I had an old Subversion server where four SVN repositories were hosted for quite some time and a few days ago I decided to nuke that in order to reduce our costs. Thank god I created a simple tar -czf of the whole directory and I know it’s not the correct way to backup Subversion repos, but I literally had no time to think and play around. Besides, I didn’t think I would ever need those again…

I was wrong, two days later I got a request for a few files that were only in that SVN repo, they never made it to our Git servers since we thought we’d never need them again. I thought this was going to be a nightmare, especially after reading some tutorials explaining how and why Subversion repos have to be backed up and restored. For a second or two I thought we lost those files forever!

Luckily, it wasn’t that bad. I had the archive copies on my local Ubuntu machine, I extracted them and tried my luck at svnadmin verify repo_dir and bing! Verified all 1700 revisions! Maybe this was a coincidence, same version of Subversion, similar operating system, … Here’s how I checked out my latest revision and got those files safe and sound:

 $ svn checkout file:///home/user/backup/svn/repo_dir .

And that worked perfectly! Took a while, but it did, so this post may come in handy, but the lessons to be learned here are: don’t use Subversion (go with Git or Mercurial), if you do use Subversion then use the correct ways and tools to backup and restore repositories (Google that for some great articles), and local repositories can be checked out with the file:/// prefix. Yeh, three slashes, and on Windows environments too ;)

Hope that helped you out. Cheers!

The Web Development Cycle Explained

If you’ve ever been to software development classes you probably already know about the software development cycle: Pre-alpha, Alpha, Beta, Release candidate (Gamma, Delta), Release to Marketing and finally, General Availability. Well it goes pretty much the same in the web development world, except that there are a few other parties involved in the whole process, and today, in the dynamic era of the Internet, web development corporations, small and medium businesses and tiny web development groups often work from different places.

I’ve been working freelance for quite a long time, and most of my projects were solo or perhaps two or three people at max, leaving me as the leading developer. Since January this year I’ve been working on a few projects with a team of over 5 people and high-demanding standards. This raised the problem of project managing and a completely different view of the web development cycle. In this article I’d like to outline the major steps of the software development cycle and how they could be applied to the web development business.

Development: Pre-alpha Stage

As we all hopefully know, each web application starts from a project overview written by both the client and the web development company. After this overview has been approved, the designers start to sketch, the programmers start to write code blocks, and the content managers gather keywords and start writing short subjects, while the client sits back and gets ready to mock anything that is sent their way. This is probably the pre-alpha stage, sometimes called wireframing or mocking up (in general, not only design terms).

Development: Alpha Stage

This, perhaps, is the stage where the design layouts are shown to the client, some text is revealed by the content managers, and something gets approved by the client. In most cases this stage takes the majority of time dedicated to the whole project. It’s pretty difficult sometimes to convince clients why certain designs look better than others, especially if they have a bad, or no taste at all. Practice shows that it’s often a good idea to:

Present the client with two variants to pick from – a really aweful one that in general terms could be used, but with a probability of at least 95% that it wouldn’t; and a really good one that you and your company placed the bet on.

Sure there’s a chance that your client goes with the first choice and you’ll end up working on something that makes you sick, but hey, this doesn’t work all the time ;)

Development: Beta Stage

After everything has been approved by the client, the Beta development stage is where coders have some fun. There’s no reason to expect that they get it right from the first time, especially when working on a big project. This is where I suggest you use the three environments method – development, testing, production. Some modern IDEs such as RubyMine (for Ruby on Rails) have built that in, allowing you to switch from one environment to another with a click of a mouse. But unfortunatelly the browser problem persists.

Web Development Cycle

I came across a really useful Firefox addon a few weeks ago, it’s called SwitchHosts. At first sight it’s something that is unnecessary – development environments could be switched to different subdomains – dev.domain.com and test.domain.com, finally the production address – domain.com. This, in my opinion, is something that’s really difficult to tackle when working with software that stores options, etc into databases. For instance – WordPress, where the database is tied to a certain domain.

Sure you could go ahead and use your text editor’s Search and Replace function but:

  • Would you really like to do it every single time?
  • Would you like to teach your other employees to do that?
  • What if there’s an error checking algorithm involved? For instance WordPress widgets disappear after a Search and Replace run.
  • It’s so much easier to keep track of a single database dump than three.

Convinced? Okay but why do you need SwitchHosts to keep one domain tied to three different addresses? Couldn’t you just use the hosts file for that? Well, sure, but then again, try to explain to your employees (I’m talking about the less coder-fellows) how to work with the hosts file, then of course they’ll forget to flush their DNS cache as soon as they’ve added that extra line. They’ll end up making changes to the production server, and you wouldn’t like that, would you?

With SwitchHosts for Firefox you could switch to as many hosts file as you setup in only a couple of clicks. Besides, it’s always there in your status bar, showing which hosts file is currently active which, believe me, solves plenty of problems. One more feature I love in SwitchHosts is that it flushes your DNS cache automatically as soon as you switch from one environment to another.

Make sure that you’ve setup version control systems (such as Subversion) on each of the environments. It’ll make it much easier when replicating from development to testing and back to development, as well as from testing to production.

The Beta Development Stage is probably the most important part to return back to when your client is not satisfied with the results, or perhaps would like to add a few minor changes to the layout, text or pictures. Of course they should be charged extra for this, and it’s extra work for you, but hey, it’s their money they’re wasting ;)

Testing: Release Candidate

Each Release Candidate should probably be located on the remote testing server, unlike the development, which should be kept locally for the developers. Of course it’s easier to do local testing, but:

  • All tests should be held in environments as close to the real world as possible, thus a remote machine, preferably on the same hosting plan as you production server. Don’t confuse “same hosting plan” and “same server” – your testing and production units should have different IP addresses.
  • Your content managers will need to polish their work, would you really like to educate them to install server software and deploy the project on their local machines, nah! ;)
  • After all our testing environment couldn’t be located on the same machine as the development. Wha? Run two different copies of Apache?

So, the RC is where you show off to the project manager, who asks the content managers to review their work on a working website. They could do this online themselves, or they could send reports to the developers.

Release to Marketing (from the general software development cycle) should be in the middle of this stage, where a decision is usually taken to go on and show the results or return back to beta development.

So if Marketing approves the whole thing, the end of the RC stage is where you should show the results of you work to the client. Of course they’ll need some hosts-file hi-jacking, and perhaps with the SwitchHosts addon it wouldn’t be as dirty as it usually is. If the client has good reasons not to do this, then of course you should drive it over to a sub-domain. But I really don’t recommend you showing off on the production server, especially if there’s an old website working there – clients don’t like their old websites to be taken down before the new one is 100% approved. Some of them don’t understand that there’s an option to restore their old website ;)

And when the Boss says yes …

Production: General Availability

This is the victory stage. It’s where both parties yell “Wohoo!”, drink champagne and drunk coders deploy the project to the production server overnight. What could possibly go wrong? Often takes less than a minute:

svn checkout svn/tags/1.0 .
mysql -uname -ppass db < include/db.sql

Great! Well this doesn’t mean that you can go ahead and delete everything you’ve done at the development environment. Yes, testing server can go to rest, but you should always keep a copy of all your work, preferably in an archived Subversion repository. Why?

  • The client could return in two weeks and say: hey, remember you showed me that second version before this third? Could we return to that if payed you some extra cash?
  • Your client could ask you for a backup copy of the work you’ve done. You shouldn’t have problems handing it over or restoring the website if they broke it within a few weeks. Keep a good reputation.
  • Parts of it (even those which were rejected) could be quite useful in future projects.
  • If clients are satisfied with the work you’ve done, they will probably return to you when they’ll need to add a few extra functions to the website that you made.

Well, that’s about it. Of course if you’re a solo-freelancer you wouldn’t need any of this stuff. You wouldn’t need version control systems at all, as all your work is done by you. A project backup is simply a copy-paste issue. Though it’s sometimes useful to imitate teamwork. After all, you can’t be sure that you’ll be coding solo for the rest of your life ;)

There are tonnes of free and open source tools out there to help you out in version control – Subversion; and in project management – The Trac Project (P.S. check out these 5 Really Useful Trac Reports).

This was the web development cycle outlined based on my experience. Sure some individuals and companies work using modified versions of this cycle, some use a completely different model. But, whichever one is picked, it all comes down to three major phases – development, testing, production. So, what’s your case? Which model is used by your company and why do you prefer to work that way? Feel free to give a word in the comments area below.

Thinking Subversion

Since I had to clean up all the mess in my SVN repo today, I’d like to show you some nifty tricks ;) First of all, I encourage all Windows users to forget about TortoiseSVN and all the other GUI versions, cause there’s nothing better than the good old plain-console subversion client. Just add it to your %PATH% system variable and you’ll be able to use svn commands from anywhere in the Windows command line.

Okay, cleaning up. During any cleanups and reordering subversion repos you’ll encounter with moving, renaming and deleting files. And here’s my favourite South Park phrase – “Just don’t mess up okay? You’re messing up too much!”. Do NOT delete, rename or move files thourgh your Windows explorer. The right way to do it is using SVN commands:

svn rename oldfile.php newfile.php
svn move oldfolderfile.php newfolderfile.php
svn remove unwantedfile.php

Yes, and the file will be removed/renamed/moved on your local working copy, then in the repo after you commit changes. Anyways, if you do mess up, you can always throw everything to your trashcan and start over with a fresh checkout (I’ve got a lot of temp folders full of these ;).

And of course property setting (propset) and removing (propdel) when it comes to svn:ignore properties. Properties in SVN are applied to the directory you pick and will not apply to its subdirectories, so be careful. Here’s how you set the svn:ignore property (for filename.php) on a folder:

svn propset svn:ignore "filename.php" folder

And here’s how you remove it:

svn propdel svn:ignore folder

That’s about all I had to say. Oh and don’t forget to cleanup sometimes:

svn cleanup

And since I’m a NetBeans fan, I just have to say that NetBeans IDE is just great when it comes to version control. It supports both CVS and SVN repos and it’s real quick at getting the job done if you split up your sources into NetBeans projects. It has an in-built repo browser, diff viewer and others. And it’s just a one-click-view-changes-and-commit ;) I love it. Go get yours: netbeans.org.