update: authors and price comparison

I’ve been very bad and not done an update on the site’s progress in a long while. There’s been a lot going on.

New Features:
I have been pretty busy updating the site. First off, as a quick win I added a Google+ button. Next, I created Author pages, which as you can guess gives you a list of all the books the site has for a particular author. It all helps.

The next big addition is that I added our first alternative book seller. The Book Depository, is the second seller to go on, so you can now compare prices, on the price comparison site, pretty handy… I also started trying to add in Tescos through their Beta API, but it only covers groceries, which I found out though much faffing. Hopefully, they will soon expand it out to include books.

The addition of new sellers lead to several quite big changes to the site. First off, the scraping script that used to be one big script is now redesigned and split out. Now you have a main class that scrapes the latest top 20 from Amazon, then you have satellite scripts triggered over HTTP for concurrency, that scrape the individual book sellers.

The second is that I have gone all newfangled and gone proper OO with the introduction of the Book Object. Now that we have multiple prices we want to be able to get the cheapest, as well as show all prices in ascending order. But we don’t want to be handling this in every controller and I didn’t feel it would work well in a helper. So I created the Book class with is instantiated in the Book model and overrides the standard CI results objects that get return per DB row. It all works pretty sweet and I have a feel of a job well coded, which is why we code in the first place (or at least I do).

Some other features that have been in for a while but I have not mentioned before are: auto updating site maps and OpenGraph on the book pages to make sharing in Facebook and other supported social media sites much easier and look better to users.

So, all in all, a lot of work, especially as it’s just one of my many side projects, but before you go and rush to check out the new features, there is some bad news. The site is broken at the moment, the scraping tool has had a fail. Since Amazon changed their markup on book pages the scraper has been failing. And due to the way it failed it has stopped the site from working. I did try and code things so that if it failed then the site should live on! But in transferring it over to the new scraper layout I must have put in a dependance by accident (that’s the trouble with coding late at night after a full
day’s work). But there is a silver lining, Amazon have released a new API.

Scraping and APIs:
When I first put the site together Amazon had deprecated their API and there was no other way then scraping the site to get the data I needed. The new API should make things a lot easier and more stable. I have started reading though the manual and I should have it all setup soon. As a side note, the new API is very powerful giving the option to have your own shopping cart on your site and passing a complete order over to Amazon to process, where as at the moment you have to send users over with a link to one product and hope they buy the other things your have recommended. I’m not going to use this feature on this is as it won’t fit the price comparison model but I will be using it for yet another side project I have in mind.

Traffic & Earnings
At the moment the traffic is not high, in fact it’s pretty dismal, about 40 monthly uniques and I have yet to make a sale. The lack of sales don’t bother me too much at the moment, as up until recently there where no prices to compare and there was an issue with prices showing £0 for a week or so, when Book Depository did not have the book listed. I would have hoped for much higher traffic by now. So if you have any suggestions on how to get traffic up let me know. I am thinking of running an Ad Sense campaign closer to Christmas, but we will see if I can get the site up to scratch in time.

What’s next?
On the author pages I’d like to get some information about the author from somewhere maybe Wikipedia. I’m open to suggestions on where I can scrape that from. Obviously I need to get the Amazon API integrated and the site back up and running properly. Then start investigating other book sellers to add in.

It’s been a bumpy ride so far, but as far as coding experiments go it’s working well. Lets hope we can get some sales in for Christmas.

Visit the Best Cookbook Price Comparison site
Previous post about
All posts

About This Post

Leave a Reply