It should be noted that how well gold adds to you portfolio diversification depends on what other assets you hold - as can be seen below, Gold showed much less covariance with the ASX300 Industrials Index that it did with the ASX300 Resources Index. So, including gold for diversification purposes would make more sense if your portfolio includes US or UK stocks, than, for example, Canadian or Australia stocks, where there is a greater exposure to the resources sector.
It is possible to buy gold bullion or gold coins and store the physical gold - but storage costs can be significant. Although there's nothing stopping you burying some gold coins in a zip-lock bag in your back-yard, Europe is littered with caches of medieval gold that was buried for safe-keeping and never recovered by the owner!
One alternative would be to buy and sell gold bullion online, for example through BullionVault.com. This company stores gold bullion in high-security Brinks vaults at various locations around the world (London, New York, Switzerland). Because the gold doesn't move it is safe, secure, cheap and easy to trade online. Client holdings are reconciled each day and published online using anonymous aliases. You need to fund your account by transfers from your bank account in order to start trading gold. New accounts get a "free" 1 g of gold credited to their account (worth around $15), but it only becomes "yours" once you fund your account and can start trading for real (you can trade your 1g for practice, but it will get forfeited if you don't fund your account within two weeks).
The main screen at BullionVault.com shows the current buy and sell prices for gold in any one of three currencies (USD, GPB or EURO) for the three vaults. The buy/sell spread seems to be around 0.4% and BullionVault.com charges a tariff of 0.8% for each buy or sell order. So the round trip cost of buying and selling bullion would be around 1.9% (There are lower tariff rates if you trade a large amount during the year).
This option may suit some investors in the US, UK or Europe. Personally I've previously bought 1 oz gold bullion "bar" from the Perth Mint and stored it at home. For larger amounts I'd trade the gold "warrants" issued by the Perth Mint that are listed on the Australian Stock Exchange (ASX code ZAUWBA) as you don't have the storage and security issues that relate to physical bullion. The only difficulty is that to trade warrants you need to have completed the required paperwork with your broker, as a normal share trading account can't trade warrants.
I personally haven't opened an account with BullionVault.com. If I was going to, I'd try to evaluate the risk of losing your investment if the company went out of business - the same sort of risk analysis you'd do before investing in, say, Prosper.com. I generally don't like funding internet purchases direct from my normal bank account, so I'd recommend opening a new bank account with just the amount you wish to invest, and use that account for use with Prosper.com, BullionVault.com etc.