Categorically Variable

Only search Categorically Variable.

Productivity Hacks for R

Data Science Productivity Hacks
A few quick productivity hacks for R.
Data Scientists are inherently productive, but there are some simple things which can make you even more prolific. I’ll briefly talk about a few hacks that not even Tim Ferriss knows about, then give a blazing fast demonstration.

One of the speakers for satRday (Cape Town) 2018 fell ill at the last minute. Fortunately I had prepared this as a backup talk.

 Read more

Restoring a Django Backup

It took me a little while to figure out the correct sequence for restoring a Django backup. If you have borked your database, this is how to put it back together.

 Read more

Extending DataGrip Evaluation

DataGrip is a great tool for accessing a wide range of databases.

You can get a free 30 day evaluation license. But perhaps you want to evaluate for a tiny bit longer?

 Read more

SQL Server from Ubuntu

Setting up the requisites to access a SQL Server database from Ubuntu.

 Read more

Installing rJava on Ubuntu

Installing the rJava package on Ubuntu is not quite as simple as most other R packages. Some quick notes on how to do it.

 Read more

Linux VM on Azure

A quick tutorial on how to create a Linux VM on Azure.

 Read more

Ethereum: Running a Node

Once you’ve installed Geth you’re ready to run your own Ethereum node.

 Read more

Ethereum: DIY Tools for Smart Contracts

What tools do you need to start working with Ethereum smart contracts?

The Solidity Online Compiler provides a quick way to experiment with smart contracts without installing any software on your machine. Another promising online alternative is Cosmo.

However at some stage you’ll probably want to put together a local Ethereum development environment. Here are some suggestions for how to do that on an Ubuntu machine.

Since I’m just feeling my way into this new domain, I’m not sure to what degree all of these are necessary. I do know for sure, that Truffle and testrpc are crucial.

 Read more

NTP: Synchronise Your Watches

Just like an old fashioned grandfather clock, time time on your computer’s clock can slowly drift. You can quickly verify the accuracy of your clock by comparing it to https://time.is/. It’s not unusual for it to be anything from a few seconds to a couple of minutes out. For most purposes this is not a major issue, but there are some applications which are very time sensitive.

NTP (Network Time Protocol) is a tool which will synchronise your computer’s clock with a network of accurate time servers, ensuring that it’s always accurate.

There’s a lot to be said about NTP, but this is a quick guide to getting it up and running on an Ubuntu machine.

 Read more

An Ethereum Package for R

Bitcoin has become synonymous with “cryptocurrency”. Ethereum is another cryptocurrency which, although not as hyped at Bitcoin, presents some attractive characteristics. The foremost of these is the ability to create sophisticated smart contracts.

This post introduces the new ether package for interacting with the Ethereum network from R.

 Read more

Moving a Running Process to screen

I am not sure how many times this has happened to me, but it’s not infrequent. I’m working on a remote session and I start a long running job. Then some time later I want to disconnect from the session but realise that if I do then the job will be killed.

I should have started job in screen or tmux!

So, is it possible to transfer the running process to screen? (Or, equally, to tmux?) Well it turns out that it is using the reptyr utility. I discovered this thanks to a LinkedIn post by Bruce Werdschinski. A slightly refinement of his process is documented below.

 Read more

Creating an Amazon Machine Image

Creating an Amazon Machine Image (AMI) makes it quick and simple to rebuild a specific EC2 setup. This post illustrates the process by creating an AMI with ethminer and NVIDIA GPU drivers. Of course you’d never use this for mining Ether because the hardware costs are still too high!

 Read more

Using Large Maps with OSRM

How to deal with large data sets in OSRM? Some quick notes on processing monster PBF files and getting them ready to serve with OSRM.

 Read more

EC2 Missing Disk Space

This morning I created a r3.xlarge spot instance on EC2. The job I’m planning on running requires a good wad of data to be uploaded, which is why I chose the r3.xlarge instance: it’s cost effective and, according to AWS, has 80 Gb of SSD storage.

So I was a little surprised when I connected to the running instance and found that the root partition was only around 8 Gb. This is what I did to claim that missing disk space.

 Read more

Variable Names: Camel Case to Underscore Delimited

A project I’m working on has a bunch of different data sources. Some of them have column names in Camel Case. Others are underscore delimited. My OCD rebels at this disarray and demands either one or the other.

If it were just a few columns and I was only going to have to do this once, then I’d probably just quickly do it by hand. But there are many columns and it’s very likely that there’ll be more data in the future and the process will need to be repeated.

Seems like something that should be easy to automate.

 Read more

Analysis of Feedback from satRday [Cape Town] 2017

We recently announced the second satRday (Cape Town) conference scheduled to take place on 17 March 2018. Obviously we want this to be bigger and better than this year’s event, so we are paying careful attention to the feedback that we received from the first event.

This is a quick analysis of the feedback. We sold 192 tickets and gave out 11 complimentaries to the event. There were 107 responses to the feedback survey, which means that we heard back from more than half of the people who attended, which is hopefully a representative sample.

 Read more

Durban Twitter Analysis

I was invited to give a talk at Digifest (Durban University of Technology) on 10 November 2017. Looking at the other speakers and talks on the programme I realised that my normal range of topics would not be suitable. I needed to do something more in line with their mission to “celebrate the creative spirit through multimedia projects from disciplines such as visual and performing arts” and to promote “collaboration across art, science and technology”. Definitely outside my current domain, but consistent with many of the things that I have been aspiring to.

To be honest, I was pleased to be invited, but when I sat down to consider what I would talk about, I found myself at a loss. I’m not currently engaged in anything that ticks many of those boxes.

But I am loathe to turn down an opportunity to speak. So I made a plan. In retrospect it was not a terribly good plan. But it was workable. I decided to speak about gauging sentiment relating to the city of Durban using data from Twitter. You can find my talk outline here. This post touches on some of my results.

 Read more

Hello Durban, tell me how you're doing!

Hello Durban, tell me how you're doing!
Everybody speaks their mind on Social Media. So, what are people saying about Durban? I'll harvest the Twitter stream to answer this question, making use of some simple analytics, sentiment analysis and machine learning.

This talk was presented at Digifest (Durban University of Technology) on 10 November 2017.

 Read more

Speaking Bucket List

A list of conferences and meetups I’d like to speak at in the next few years.

 Read more

Installing NVIDIA Graphics Driver on Ubuntu

Recipe for installing the NVIDIA binary drivers on Ubuntu.

 Read more

Categorically Variable