MonthAugust 2013

“Wine Support” for LibreOffice

Sorry about the “misleading” title.

I mean wine, not wine ;-)

A few weeks ago I saw an interesting sponsorship on LibreOffice.
I took a look at the sponsor profile – Bill Robertson, which took me to his website – www.debortoli.com.au – which is an Australian company that makes and sells wine.
I thought that was interesting because that’s just not the average “corporate visitor” you see on FreedomSponsors. Most of our users come from IT companies.

After some googling I thought De Bortoli Wines was very cool, and it would be nice to contact Mr. Robertson and maybe get an interview from him.

He kindly agreed and here it is!

Can you tell a little bit about yourself and what you do at De Bortoli Wines?

I am a local of the Murrumbidgee Irrigation Area, having lived here most of my life. I have held a variety of IT positions, at several agri-business companies, and like to call myself an IT generalist. I joined De Bortoli Wines in 1995 and moved into the position of IT Manager in 1998.

I have very much  enjoyed the challenges and strategy involved with this executive role. Some of the more exciting projects we have implemented include our Linux Desktop rollout (circa 1995), our “Pentaho” Open Source BI implementation, our custom HTML5 Visual scheduling application (for bottling line scheduling), and our smartphone and tablet rollout (complete with a custom Salesforce app!).

Can you tell some facts about the company, like what size and how old it is?

De Bortoli Wines is a third generation family owned wine business founded in 1928. The Executive Directors are the grandchildren of founder Vittorio De Bortoli, and Darren De Bortoli is our Managing Director.

The business crushes around 80,000 tonnes of grapes and produces a large variety of wines, which are distributed all around Australia, as well as being exported to many major centres around the world. There are two main wineries – one is located alongside our Head Office near the regional NSW city of Griffith, and the other is located in the beautiful countryside of the Yarra region of Victoria and includes our award winning restaurant.

How important is the IT department role in De Bortoli’s operations? How is software important for making and selling wine?

As one of Australia’s largest wine companies there is a lot of complexity to manage – multiple vineyards, wineries and distribution centres, with IT being involved in many of the activities. At an overview level we have systems to purchase grapes and schedule grape deliveries, document winemaking processes, schedule bottling, manage logistics, manage sales (including EDI and tablet based order entry for sales force).

We provide support for our online activities (Websites, Blogs, and B2C activities), EDI (B2B), as well as all the standard ERP style functions (Payables, Receivables, Costing, Inventory, Forecasting, MRP II, DRP, GL and BI, etc).

On the tactical side, we support nine networked sites in Australia (LANs and WAN), several hundred desktop computers (vast majority running single image Linux based on Ubuntu), the usual printer/scanner fleet, and an extensive Android smartphone and tablet fleet.

How much does De Bortoli Wines rely on Free/Open Source Software? Can you give a list of what software you guys use and what for?

Our main focus is on Open Standards, which helps us avoid lock-in, and provides maximum flexibility while minimising long term costs. Our Open Standards focus has often (though not exclusively) led us towards Open Source solutions.

Some Open Source examples include:
* Linux clients (single image desktops running on various flavours of Linux since 2004)
* SAMBA (utilised for over a decade for fileserving and Domain control)
* Red Hat’s 389 LDAP server (for user management)
* Pentaho (ROLAP BI solution)
* Pentaho PDI’s “Kettle” (ETL tool for the majority of our data transformation and integration tasks)
* OCS Inventory (computer inventory system)
* Nagios/Nagvis/Cacti (monitoring)
* TYPO3 (CMS for intranet, extranet, Internet sites)
* Magento
* dotProject
* Shell in a Box (browser based ssh)
* Guacamole (HTML5 clientless remote desktop)
* Linux servers (RedHat and variants such as CentOS and Scientific Linux)
* MySQL/Postgres/PHP/Python/Java/Apache/Tomcat etc

Our Open Source desktop software (Linux and Windows) includes:
* FireFox (since around 2004)
* OpenOffice (also since 2004/2005)
* LibreOffice (currently piloting)
* Inkscape and Gimp
* FileZilla, VLC etc.

And lots and lots of other random stuff such as Eclipse, PhoneGap, JQuery, Scite, JEdit, VNC, “putty”, “Ghostscript” etc etc

Where did you hear about FreedomSponsors?

When we mentioned on a LibreOffice bug report that we would be willing to help sponsor a fix, FreedomSponsors was suggested as a mechanism.

Do you have any suggestion about how FreedomSponsors can help companies like De Bortoli? What else should we do? :-)

I think the main challenge will be to connect all the potential developers with the potential sponsors which will just take time to build the connections.

And of course, spread the word we have bounty on a LibreOffice password bug! (http://www.freedomsponsors.org/core/issue/316/allow-filesave-with-password-for-xlsx-and-other-not-libreoffice-xml-based-formats)

Please leave a ending message for our readers!

The great thing about Open Source is the community, and its nice to have a chance to say thanks. So, Thank you all :)

Thank you Mr. Robertson. I wish success to you and De Bortoli Wines. Please keep supporting Free Software!

Using Dropbox to trigger deploys.

TL;DR:

echo $RANDOM > ~/Dropbox/fstrigger/deploy

If I run the command above on my machine, what does it do?

- It will write a random number to a file, duh.

Ok, it does that. But it also causes the FreedomSponsors application to update itself from git. About 20 seconds later, I get an email from the server with the deployment logs.
No need to ssh into the server anymore, yay :-)

Now the whole story…

While I was configuring uwsgi for FreedomSponsors (more info in the previous post), I learned about uwsgi emperor mode. From the awesome uwsgi tutorial:

uWSGI can run in ‘emperor’ mode. In this mode it keeps an eye on a directory of uWSGI config files, and will spawn instances (‘vassals’) for each one it finds.
Whenever a config file is amended, the emperor will automatically restart the vassal.

I really liked the way this emperor mode worked, because when I change the python code, there’s no need to restart the web server. I just need to “touch” the uwsgi config file:

touch ~/frespo_uwsgi.ini

This is what gave me the idea to remotely trigger the deployments to FreedomSponsors using Dropbox.

This is how it works:

1) The server has Dropbox installed, and it has its own dropbox account.
2) That Dropbox account has a “fstrigger” directory, which is shared with my personal Dropbox account.
3) The server monitors the file: ~/Dropbox/fstrigger/deploy, so that any changes to it will trigger the deployment script
4) The deployment script:
- updates the code from git
- perform a database backup
- install the new code and respawn the uwsgi workers (by touching the uwsgi config file)
- sends an email with the deployment log to me.
5) After I push some code to git, all I have to do to make it go to production is to change the contents of ~/Dropbox/fstrigger/deploy (on my machine). Dropbox sync will kick in and trigger the deploy on the server.

Cool huh :-)
I can imagine an entire farm of servers self updating simultaneosly with a similar setup.

So, if you want to reproduce this setup, here’s a few tips:

Install the command line version of Dropbox on your server.

The instructions here should be enough:

https://www.dropbox.com/install?os=lnx

You may need to edit your /etc/rc.local to make sure Dropbox will run at boot time.
Here’s what it looks like in our server:

sudo -u frespo /opt/dropbox-dist/dropboxd &

Now you need something to monitor file changes.

incron

There’s a tool for that:

apt-get install incron
# But don't do this yet

This is a unix service like cron – the difference is you can bind scripts to file events.
After installed, you can

incrontab -e
# edit the incron table
incrontab -l
# show the incron table

There’s a problem though.
During my testes with incron, the script would trigger if I changed the file locally (on the server), but it wouldn’t trigger when the file changed via Dropbox. I have no idea why (maybe I should file a bug for that)

But then there’s inotify-tools

apt-get install inotify-tools

This is a package with a inotifywait utility command that waits for file events.
(And this I could get to work over Dropbox file changes)
So, I have a deployWatch.sh script like:

while inotifywait /home/frespo/Dropbox/fstrigger/deploy; do
 /home/frespo/deployAndRestart.sh
done

And, I also put that on /etc/rc.local:

sudo -u frespo /home/frespo/deployWatch.sh &

And there you have it! A Dropbox-triggered self updating website :-)

If you have any comments to improve our production environment, please let us know in the comments below!

Improvements on the production environment

We recently made a few changes to our production infrastucture, detailed below.

Thank you AWS, Hello DigitalOcean

Since last saturday, we moved from Amazon AWS to DigitalOcean.

Previous environment:
Django over wsgi served via Apache, with a Postgres database, all running on a single micro AWS instance.

With this setup, FS was able to serve 2 requests/second.

New environment:
Django over uwsgi, served via nginx, with a Postgres database, all running on a single $10 “popular plan” machine (1core, 1Gb memory)

With this setup, FS was able to serve 5 requests/second.

Of course, the goal was not to benchmark DO x AWS.
But I do have the felling that DO would win on a “more fair” fight.

Also, the DO instance boots ridiculously fast – in my experience, AWS instances have much longer boot times.
Overall I’m very happy with the switch, but we need to see how it will behave in the long term.
I’ll keep the infrastucture on AWS for a while, just in case.

New URL

We also changed the base URL: freedomsponsors.org instead of www.freedomsponsors.org.
That’s shorter and nicer.

Web server setup

The nginx+uwsgi setup was very straightforward thanks to this tutorial by Daniele Procida.
Really, it’s one the best tutorials I’ve seen in my life.
Thanks Daniele. Keep up the good work!

What I liked most about uwsgi is that I don’t need to restart nginx to deploy a new version.
I just need to touch the uwsgi config file (touch ~/frespo_uwsgi.conf) and uwsgi will respawn the workers – which happens very fast (less then 1 second).

Caching

A friend told me I should try and configure caching to see how much performance improvement I would get.
Now FS runs with a memcached cache. Django talks with the cache using a standard unix (file) socket.
I did a benchmark running “ab -c 10 -n500 http://freedomsponsors.org/” on my machine, and got 18 requests/s as a result – but the CPU on the server was still below 10%, indicating that the limiting factor was the network.
So I ran the benchmark on the server… and got 616 requests/sec
Now that’s nice!
No need to worry if FS makes it to front page on Hackernews :-)

Emails

On the previous setup, email was delivered with a local postfix installation.
Now, I still use postfix, but I’m using Mailgun as a relay host.

Database backups

No change here. A postgres dump is saved to Dropbox (and syncd with my computer) every 12 hours.

Deploying

I also changed the way deploys are made. I don’t need to ssh into the machine for deployments anymore.
I’ll talk more about that in another post.

That’s it.
If you have any suggestions to improve our production environment, let us know in the comments below.

FreedomSponsors Bugzilla Plugin

Hi everyone.

I’d like to announce the availability of the new freedomsponsors-bugzilla-plugin.

This plugin for Bugzilla works in the same way as the Jira plugin (that is used by Jenkins): it will add a nice, discrete “Sponsor this” link on your Bugzilla’s bug pages, just below the “Importance” bit.

The link leads to a FreedomSponsors page where the user can sponsor that bug.

If you’re a project owner running a Bugzilla installation, you might want to consider installing this plugin if you’re interested in attracting sponsorships to your project.

See more details on Github.

This plugin was developed by Marc Schumann, and its development was sponsored via freedomsponsors :-).
Thank you Marc, for doing it so well and quickly!

Cheers!
Tony Lâmpada

© 2017 FreedomSponsors

Theme by Anders NorenUp ↑

Follow

Get every new post on this blog delivered to your Inbox.

Join other followers: