Blog

Giving your first machine learning workshop

Last week I offered to give a workshop on machine learning to share some of what I’ve learned so far. I had 2 goals for my workshop – first, I wanted to show everyone that they shouldn’t be afraid of machine learning. Second, I wanted to get people set up with anaconda so they can continue to learn themselves.

I had a cross functional group of ~10 colleagues – business analysts, data analysts, UX designers, developers and test analysts. It went quite well! Everyone seemed engaged for the hour we spent together. I ran a survey after the workshop and achieved an NPS of 63!

Survey results

If you want to give a machine learning workshop please feel free to use this a template. I used resources already available online for my workshop. I only made some small changes and I have included all the links and files below.

Some lessons for you to apply

  1. As a facilitator I wouldn’t recommend having more than 12 attendees.
  2. Be very clear on what you’re going to cover at the start
  3. If you have time, pick a problem in your work domain. But don’t sweat if you can’t.
  4. Use visual learning tools. There are some great ones for decision trees and random forests.
  5. Don’t make people write code the first time out. Just have them execute lines.

The process

 

Install Anaconda

Before the workshop, I asked everyone to install Anaconda on their computers from https://www.continuum.io/downloads. I decided to use python because it’s what our data team use and enables future model sharing.

Explain the algorithm

I spent maybe 20 minutes explaining decision trees with NO MATH using the incredible visualizations on http://www.r2d3.us/visual-intro-to-machine-learning-part-1/. This page is beautiful and self explanatory.

Visual guide to decision trees

Downloading the files

I used a popular titanic data set from kaggle.com. I only used the training set for both training and testing to keep it simple. You can download the files here: https://www.darraghoriordan.com/wp-content/uploads/2017/06/titanic-data-and-solution.zip

Run through the code

I used an excellent kernel from https://www.kaggle.com/helgejo/an-interactive-data-science-tutorial. I simplified it by only using three features for my model. Some of my visualizations are from super data science’s machine learning A-Z course.

With all this in place I just asked people to open Spyder from Anaconda. They had to set the working directory to where they had unzipped the files and restart the kernel.

Then we stepped through each section together. We talked about all the visualizations and results.

Stepping through spyder

I really tried to stress that training the model is easy but understanding and preparing the data is difficult.

Identifying important variables

 

I got some great feedback on the machine learning workshop. Some attendees wanted to stay longer to get better accuracy and learn more algorithms!

One person wanted a problem more closely related to our work so I’m going to do that next time.

If I had more time I would try to form groups and have them compete to get the best accuracy. As facilitator I would be walking around trying to help them out individually. Instead this was ‘homework’.

Accelerating machine learning adoption

Last week I had an amazing opportunity to present about machine learning and innovation to 110 people leaders at my company. It was a mixed audience of technical and non technical people.

My message is that machine learning needs to be understood by everyone in the business, not just the data scientists. Domain knowledge with machine learning will really enable successful data projects.

Here are my slides and some notes

A few months ago I followed a tutorial on http://course.fast.ai and entered a Kaggle.com competition. The competition problem was to label 20,000 images of cats and dogs using computer vision. There were 1300 entries from around the world.

I downloaded an existing, freely available algorithm – vgg16 and slightly modified it to get 87% accuracy.

I thought this was a great result. It’s REALLY hard for computers to tell what is in these pictures. 5 years ago a team of scientists got 57% accuracy with this same dataset.

However it turns out I didn’t do very well:) I came 600th! The winners got closer to 97% accuracy.

We’ll never all be data scientists but the technology is at a point where anyone new to machine learning can download great solutions and start solving these problems. You as domain experts are in the best place to see these opportunities and start experimenting.

What is machine learning and why is different to what we do now? This is very simplistic but with traditional computing we would tell the computer exactly what result we wanted for a given set of inputs. With machine learning we give the computer a large amount of information and we ask the computer to give us insights in to the data.

We don’t write explicit programs. The ‘program’ is an output from the data and will change based on the data.

It does this using some well known and well studied mathematics. Data scientists even have a cheat sheet for which algorithm to use. For more difficult aspects of machine learning like Deep Learning, there are some very good models available for free online. I downloaded one of these for cats and dogs.

But we’re not here to learn the cheat sheet so forget about the detail.

Just remember that the algorithms are well known for a given problem.

What gives companies an advantage in machine learning is their data.

We have an incredible set of users here. And they’re giving us some great data. Crunching all this data costs money.

One of the reasons you hear a lot about machine learning recently is that computing power has gotten very cheap. I spent just $150 for a few hours of computing from amazon for cats and dogs.

Why now? Exponential innovation…

Every few years for the past 100 years the amount of computing power you can buy for $1000 dollars has doubled. We are just at the tail end of the most recent technology advance – semi conductors. This pattern means that right now for roughly $1000 dollars you can purchase the same amount of computing power as a mouse’s brain.

If this trajectory continues then by 2024 for that same $1000 dollars you will be able to purchase the same amount of computing power as the human brain.

Now this is a wacky idea and I don’t believe it myself. But that’s perfectly normal! Humans are really bad at thinking exponentially.

 

If I ask you to walk 30 steps linearly then that’s easy to picture, 30 meters. However if I ask you to walk 30 steps exponentially, doubling every step – 1m, 2m, 4m, 8m. Then by the 30th step you will step billions of meters. The final step will take you 26 times around the world!

We can’t think this way but this is how fast and how cheap computers are becoming.

There are three things we can do to help accelerate adoption of machine learning throughout the business.

Realize that machine learning is absolutely accessible and it’s not magic once you know what types of problem can be solved.

There are 5 major types of problem…

v
Classification
Regression
Regression
Clustering
Clustering
Ranking
Ranking
Anomaly Detection
Anomaly Detection

Think about problems in your part of the business that can be phrased this way.

We need to collect better data, not just more data. We need to collect relevant data and this is where your domain knowledge is vital.

You are also in the perfect position to identify gaps in our current data. We should find these as soon as possible and start plugging them.

We need to identify any possible external sources of data, council data for example.

You should identify areas of the business where we are making subjective decisions. If we can eliminate ambiguity and subjective decisions from the business we can make better decisions.

Collaborate with data scientists – your domain knowledge combined with the skills of our data science is what will produce the best results

Don’t silo data. ask your data team where you should push data so everyone in the business can access it.

Don’t be afraid of sensitive data. We can anonymise the data and still get great insights from it.

We have some slack channels. Anyone can join the channels. We are all learning and these are a safe space for any level of knowledge in machine learning.

 

We will be running classes ranging from this type of over view information all the way to implementing real solutions.

So don’t be afraid of machine learning. Here we have a huge loyal user base and they’re generating amazing data, we have a group of the top technical and business talent in the country.

But our industry is changing faster than we can imagine and we need to use every tool available to keep our advantage in the future.

Think of machine learning as another technology or tool like Word, Excel or Photoshop. Learn about it. Get involved.

Please get in touch if you would like more information.

Some images from this amazing article on wait, but why: http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html

The slides and idea for exponential innovation from this great talk by Kaila Colbin: https://www.youtube.com/watch?v=XwxwVSJcOGU

Course: http://course.fast.ai

Startup Weekend Auckland May 2017

Wow, it’s a week after Startup Weekend Auckland. I’ve had time to get my thoughts straight about the whole experience. I learned a hell of a lot of positive lessons about idea validation, about myself and about working in teams.

Friday Night

We started the event on Friday night with a brief introduction to the format and some key organizers. Thirty enthusiastic 60 second pitches quickly followed. There were ideas for B2B services, tournament platforms, monetizing Facebook, recipes and shopping.

Andres and Billie Friday night
Andres and Billie Friday night

The problem that resonated with Billie, Andres and I was presented by Steffi who spoke about infertility and how it could be reduced with simple education. I was interested because I know a family that is affected by infertility and it was the only idea that wasn’t immediately commercial.

After the pitches we went out to meet Steffi and ask her if she would like to have us on her team. Our team was quickly joined by Andy who works in property and Ally who works in marketing. By the end of the voting we had a very well rounded team.

We stayed at the event until 1am Friday night researching the idea. We learned about existing solutions and tried to figure out which part of the huge demographic we would try to target. It was a productive evening learning about the problem.

Saturday

We got to the second day and immediately got in to working out an informational site. We got strongly challenged by the mentors about not thinking through the greater problem.

We got some great feedback on the first pitch. One mentor warned that saying “ins and outs” may cause laughter issues” – We would turn this in to a positive by the final pitch.

Re finances – “If you can only crowdfund $3,000 in the first year no one gives a shit”. I didn’t think this way before the weekend but it’s so true. $3,000 these days isn’t much at all.

Another mentor said we should have more optimism – “What beyond a website would help these people?”. This really resonated with me and got me thinking that an informational website wasn’t a great idea.

Some problems started to develop in our group. There were passive aggressive comments around the table and it was a terrible environment. I found it very difficult to confront strangers. This made me realize how great it is to work in such an honest, direct workplace at Trade me.

Lesson 1 – There’s no time for bullshit, be honest and up front

At around 3pm on Saturday I lost passion for the informational site. I felt like any criticisms of it weren’t being listened to so I decided to use the time to learn some technology and just settled in to coding a template Billie had designed.

Lesson 2 – The person who pitched doesn’t own the idea any more, give up your plan and let the idea evolve

Our second pitch didn’t go very well because we had run out of energy and enthusiasm. The judges called us out on it and we all felt pretty miserable coming out. We left at 11pm absolutely drained, physically and emotionally.

There were some wonderfully positive outcomes from Saturday. I learned about the fertility problem in general and I learned I could work well with Billie.

Sunday

I started the last day feeling a bit apprehensive about the whole weekend but Steffi had really opened up to changing the plan and the rest of the team were keen too.

We went back to basics and started an ideation session in one of the lecture theaters. I had some ideas but mostly I just kept firm about the not so good ideas that were being bandied about. We came up with a range of ideas but didn’t settle on one for the morning pitch. It was really hard tearing up everyone’s ideas but I’m glad I did to get us where we eventually got to.

Andy gave a great pitch on what we had done for the whole morning and where we were going with ideas. The mentors saw that our energy was back and helped us out by giving us the Logic Model.

We used this model and realized we would have a better impact by focusing on young people. We argued back and forth about what to actually build while hanging out around a table outside. There were sperm iOS games and egg tamogotchi ideas. After lots of thinking we thought of a Movember clone – a month of no sex and named it Sexless September.

I thought this was an awesome idea and felt I had been negative enough all morning so after one final argument with Steffi to not have too much information on a Movember type site I checked out and got some lunch.

The team was buzzing again and I quickly built a site that Billie had designed in square space. I was surprised how awesome and easy squarespace is.

Sexless September site design

Lesson 3 – Persevere, It will come together at the end

We pulled it together by the end. We announced the new site on our Facebook pages and got 60 or so signups to our email list by the end of the weekend.

Steffi and Andy gave an incredible pitch. It started off quite somber and then Andy brought in the humorous side. What really struck me was the audience was our intended target audience and they LOVED it. Billie’s tee shirt idea went down a treat.

Giving the final pitch
Giving the final pitch

So overall the weekend was like a roller-coaster. It started high with a surprise not for profit project, dropped very low on Saturday, climbed Sunday morning with our ability to change output, dropped off again for me having to be negative to focus the output and finally finished on a super high because of where we got to.

I would highly recommend going to a startup weekend. It opened my eyes to many more of the factors involved in building a product. The mentors were awesome. Our team was awesome and the problem we had to solve was awesome. It was organised very well. It was a pressure cooker but I came out better for having gone.

Create-react-app react-router and Heroku: The 404 gotcha

I’ve been putting together a React app and I needed cheap, fast hosting so I went to put it on Heroku.

The community have put together a build pack for create-react-app. You can basically deploy in 2 minutes. It’s magic! See the instructions here: https://blog.heroku.com/deploying-react-with-zero-configuration.

Assuming you have heroku configured the tl;dr is…

npm install -g create-react-app
create-react-app my-app
cd my-app
git init
heroku create -b https://github.com/mars/create-react-app-buildpack.git
git add .
git commit -m "react-create-app on Heroku"
git push heroku master
heroku open

However! If you use react router you will get a 404 from nginx when you try to directly open a route.

The heroku build pack deploys your site on heroku as a static application so you have to tell it that for ALL routes, it should just go to your root document so react can handle the routing. The default root document in create-react-app aps is index.html

So to fix add a file called “static.json” to your root folder (next to packages.json in create-react-app) with the following configuration.

{
  "root": "build/",
  "clean_urls": false,
  "routes": {
    "/**": "index.html"
  }
}

Deep Learning: Entering my first kaggle competition

Three weeks ago I discovered the fast.ai course. They offer a practical deep learning course for coders where you get straight in to building neural networks. They provide a huge amount of support through wikis, forums, aws scripts and tooling, videos and book recommendations.

AI competitions

The first step in this AI course is building an neural network for detecting cats and dogs in images using keras. To make this easy you use an existing model called Vgg16. Yea I had no clue what any of those things were either. There is so much information in this first lesson. It took me three weeks part-time to get through it all properly, and I still didn’t go deep in to the maths.

kaggle.com is a site that lists AI competitions (with serious prize money!). Anyone can enter, but you will be up against team from Baidu, Facebook and Google 🙂 Vgg16 is a 16 layer model that won a competition in 2014 for image classification.

To do the calculations quickly you’ll need a computer with a GPU. I used AWS for this. The fast.ai course has some neat scripts that set up your EC2 instance and installs Anaconda. You need to go through an approvals process to get the AWS instance with a GPU.

Keras

Keras is a (high-level) library that sits on top of the TensorFlow or Theano libraries. It abstracts away the details of creating modular models that can easily be implemented, modified, trained and verified. It makes it easy to run your model on CPU or GPU and change the underlying deep learning library.

There is a command line tool for pulling down keras competition data.

pip install kaggle-cli

You’ll need to accept the T&Cs for the competition. You’ll need to do a forgot password if you signed up for kaggle.com with an OAuth provider. Use this password for the kg client.

Once you have the cli you download the data for the competition from kaggle. You have to put it into folders labeled cats and dogs because vgg uses folders to classify rather than filename. You have to follow the videos for all the details.

Finally you can run the following few lines and get a 97% accuracy. And that’s only from one learning epoch! This isn’t a great score these days and so they go in to refining the model further to get 99%+.

batch_size=64
no_of_epochs=1
# do it
vgg = Vgg16()
# Grab a few images at a time for training and validation.
# NB: They must be in subdirectories named based on their category
train_batches = vgg.get_batches(train_path, batch_size=batch_size)
val_batches = vgg.get_batches(valid_path, batch_size=batch_size*2)
vgg.finetune(train_batches)
vgg.model.optimizer.lr = 0.01
vgg.fit(train_batches, val_batches, nb_epoch=no_of_epochs)

Found 23000 images belonging to 2 classes.
Found 2000 images belonging to 2 classes.
Epoch 1/1
23000/23000 [==============================] - 654s - loss: 0.3473 - acc: 0.9701 - val_loss: 0.1633 - val_acc: 0.9865

I entered an AI comp!

 

Kaggle DogsCats entry

The age of applied deep learning

Ok, so I have glossed over hours of learning new things and faffing about with aws, tmux, convolutions and back propagation to name a few of the things I hadn’t much experience with.

But just the fact that I can create a working model like this, trained to a specific categorization problem is something I never imagined doing. 99% of this is me using models developed by really smart people but that’s what most developers do every single day. We take libraries and tools and use them to ship business value.

For the first time it really feels like AI is now part of the developer’s toolkit.

get started now!… http://fast.ai

Fuzzy friday S02E07

From lunch to acquisition, how Atlassian bought Trello .

Success stories from developers who started at age 35,40,45 .

Why Basecamp chose profit over venture capital.

How do you measure leadership? from YCombinator.

Folk musician Jesse had his Youtube account hacked and lost all hist fans and likes. No recourse from Google so he’s starting again. The power of internet giants to have huge effects on people’s is only growing.

Enjoy and thanks for reading!
dar

Subscribe here: https://tinyletter.com/darraghor

Webstock 2017: A long summary

Web Stock is tech conference like no other, held in Wellington, New Zealand. Once again I left feeling energized and impassioned after listening to all the speakers on such a wide range of subjects.

These are my summaries and don’t even come close to describing the full scope and quality of each talk. Use only as a guide to hear more. I’ll add links when the videos are released. I also may have made mistakes transcribing this because my hand writing is terrible!

Day 1

Tim Kadlec (@tkadlec) – Unseen issues behind web development

There are parts of web development that we don’t always consider – access, security, performance. They remain “unseen”.

There are barriers to internet access you might not consider

  • Content – e.g. access to certain content is simply blocked in many countries
  • Culture – e.g. in parts of India women cannot own mobile devices because it is described as “dangerous to their well being”
  • Cost – e.g. see GNI below. Some refugees in Europe have to decide on contacting family members at home country or eating that day.
  • Infrastructure – e.g. Only one in five Tanzanians have access to power and that power is not reliable.

53% of people leave your page if load time is >3 seconds.
https://whatdoesmysitecost.com/ will tell you how much of GDI it costs someone in poorer places to view your web site.
https://observatory.mozilla.org/ will scan your site for security issues
https://chrome.google.com/webstore/detail/axe/lhdoppojpmngadmnindnejefpokejbdd Axe can be used to check for accessibility issues

Not providing basic accessibility cost Target $6 million in a lawsuit. Basic stuff – no alt text, image maps, not using header structure.

Kim Goodwin (@kimgoodwin) – Use scenarios and storyboards

A scenario is “A plausible future story following a real person or persona from start to finish”.

We tend focus on stories in pieces of functionality but there are UX ‘purgatories’ in between our stories where the user is left hanging. i.e. Who owns the user when they are between organisational areas of responsibility.

User stories cause narrow thinking – we lose the big picture, what is the user trying to achieve e..g “User logs in” – a user never thinks, “shit I must log in to Facebook” they are always transitioning to something else.

Kim recommends lots of user discovery.

Meeting expectations in your scenario is not good enough. You should add something exceptionally good.

What would a thoughtful human do? Went through a detailed example of booking a flight using a personal assistant rather than an application. Think about how this experience would compare to your website. Spoke about being pragmatic when using these tools and spoke about obviously removing impossible things and converging on the parts of the scenario that you control or influence.

Nice simple graphic of thinking about a solution framed as “What can I provide to the next step in this process?”.

Lara Hogan  (@lara_hogan) – Performance

Gave lots of practical performance optimizations you can make.

We shouldn’t have performance cops. Everyone should think about performance.

She uses live side by side comparisons of their site loading along side a competitors to inspire people and their executive.

Also uses side by side live loading of the website on a large monitor screen for major global locations. They server the site from a New Jersey data center and this highlights how slow site load in Australia is compared to what their engineers see in New York.

Katie Dill (@lil_dill) – Considering UX on parts of the user journey you don’t directly control

Lots of positive examples of good UX from some New Zealand companies.

http://www.betterbydesign.org.nz/why-design-matters/nz-design-stories/air-new-zealand/

Questioning how you control the chaos in the parts of peer-to-peer products you don’t directly control. E.g. Air bnb hosts meeting their guests. Trade me sellers handling a purchase.

See the big picture – find out the whole user journey. Identify the parts you don’t control. Set the stage – set expectations. Make sure the user knows what they’re getting in the peer to peer transaction (Air BnB have done lots of research into what the user wants to see first on a listing detail).

Give the peers guard rails – prevent any of the peers from inadvertently creating shit experiences – used trademe seller guides as example here. Said to quiet YOUR brand here. E.g. on the Etsy store the peer sellers voice is loud, your brand is quiet. On air bnb profiles the Airbnb brand is quiet so the peer hosts personality is loud.

Keep it real – let peers be authentic. Example of uber ride where she was having a terrible day and the guy said “can I cheer you up?”. He turned on disco ball and some techno and made her day. Also gave examples of allowing video for Airbnb’s new travel experiences product rather than images because it would show more of the host’s personality and authenticity of the complex experience.

Arm your partners – Give the peers feed back. Showed the dashboard for air bnb hosts that offers suggestions how to make more money and help videos etc.

Some examples from listening to Car Talk – http://www.cartalk.com/

Lisa Welchman (@lwelchman) – Being human in the digital age

This was about Staying Human. Lots of paraphrasing on my part here. She was far more poetic 🙂

While focusing on the details (font size, some library) don’t forget that when working on websites you are creating the most profound experience humanity has had so far.

Through the internet we are bringing a new age into being. It will be a reflection of ourselves and our beliefs (so don’t fuck it up).

Some mobile applications the UN produces directly saves lives. The technology you build can directly save lives.

Introduced me to Vint Cerf through some quotes.

We are creating new and replacing the existing systems and realities (political, shopping, health, everything). We can make them good or bad.

Don’t feel helpless about bad things happening in the world. We are on the cusp of an enormous shift in humanity. You have agency, you have power. In building the internet you have influence. Change won’t come without setbacks. It brings the worst people out of the woodwork because they are afraid their way of the world is being taken away. Keep changing the world.

Marcin Wichary (@mwichary) – Three examples with a surprising detail in their UX

Speaks about one of Charles Babbage’s machines, the pacman game and mouse pointers on text mode displays. How they all have incredible UX but it’s not well known.

  • The babbage machine had error states to show people that calculating machines were safe.
  • Pacman had incredible tweaks to make the game more friendly than anything that had come before.
  • Detailed how much time and effort went into a DOS program to have a smooth mouse movement. No technical need for it but it made the physical mouse disappear and the user’s hand became directly linked to the pointer.

Very entertaining and informative!

Sacha Judd (@szechuan) – If you look closely there are technical experts in places you might not expect them

I think this was the highlight of the day for me. Excellent content and delivery. Full talk here: http://www.sachajudd.com/one-direction.

In general it was about diversity. It was themed around the fandom that One Direction has and how this has inspired non-coders to become self taught coders and media content producers (editing tumblr and wrdpress themes), movie editors, CSS editing, social media management, writing fan fiction. How all these people are perfect candidates for tech jobs but haven’t followed that traditional path.

She became part of the community,  gained some trust and then ran a survey. They are typically young women, they are full of self doubt – no formal education, didn’t know about tech roles – even the job titles, were told they shouldn’t go for tech roles.

Recommends hiring on ability (not just academic achievement)

Gave incredible examples of the things one direction fans did. They organised 55 million social media fans to basically get a single they wanted released.

Recommended http://projectinclude.org/
https://textio.com/ – remove gender specific keywords
https://triplebyte.com/ – achievement based hiring (no resumes)

Ashley Nelson-Hornstein (@ashleynh) – Humanities x Technology

Apple ads never have tech specs. Focus on the experience. https://www.youtube.com/watch?v=A_qOUyXCrEM

Big homage to steve jobs. He was at the crossroads of tech and humanities.

Gave an example about Polaroid. Inventor removed complexity of taking pictures so people could focus on photography.

Never Alone video game took Iñupiaq legends so that kids who were being lost to technology were able to re-engage with culture – http://neveralonegame.com/. Spent years working directly with elders to make sure game matched their expectations.

Cal Henderson (@iamcal) – Emojis

Fascinating history and how they are displayed on various devices. Such a fantastic speaker. Highly recommend this one!

Patricia Moore

Unfortunately Patti was hit by a car in New Zealand but she still recorded a talk from hospital while waiting for an operation. She spoke about designing for humanity.

Day 2

Jonathon Colman (@jcolman) – Wicked Ambiguity and UX

It doesn’t matter whether your a developer, tester, designer – we’re all working to prevent ambiguity in the experiences we create.

Introduced Tame vs Wicked problems.

Some examples of wicked problems – Designing to communicate with aliens https://en.wikipedia.org/wiki/Voyager_Golden_Record and designing to warn people in 10,000 years about radiation dumps.

We should embrace wicked problems. Thy ignite creativity and hep us to innovate.

Jared Spool (@jmspool) – Becoming an organisation infused with UX from top to bottom

Lots of practical advice on the various stages of how an organisation incorporates UX in to their systems and processes. Great talk. Highly recommended.

Spoke about Disney. How in 90’s their website UX was terrible. So bad user’s couldn’t tell which park they were booking a hotel for. Described the transformation from that company to the company that created the Disney magic band – a $1 billion UX project.

Your goal should be to have your entire team CEO down to individual contributors fluent in UX.

Some plays for getting your organisation to become design driven include…

Immersive Exposure. Understanding the user experience provides understanding of customer problems. Recommends everyone spends at minimum 2 hours every six weeks with customers doing UX testing. Have senior leaders listen in on customer tests and plot the customer happiness on a graph as they go through. Help get leaders to empathize.

Shared Experience Vision. Create an experience vision. Recommends 5 years ahead. Use to provide guidance for critical design decisions. Make sure everyone has a goal to march towards.

A Culture of Continuous Learning. Be retrospective.”Why did we fail?”. “What did we learn?”. Add a new item to every standup – “What did I learn yesterday?” Log the learning somewhere like a slack channel.

Lindsay Aitchison – Designing working spacesuits vs Hollywood spacesuit design

Lots of information about why spacesuits are designed the way they are and some of the difficulties and considerations for designing the Mars spacesuit.

Learned about perchrolates and why they’re going to be a problem on mars.

Janine Gianfredi (@alohaj9) – Bringing agile to government IT

Obama brought private sector technical experts in to government to fix healthcare.gov. This was very successful so they started the US Digital Service to bring private sector methodologies and people in to government. Some points for successfully introducing these methods to government were…

  • Hire and empower great people
  • Find the truth tell the truth – you have to be honest up the chain about the issues your facing.
  • Optimize for results not optics – press conferences and quarterly reports shouldnt drive the work being done. Only focus on fixing the real issues.
  • Go where the work is – break down walls. Talk to your customers.
  • Design with users, not for them – There was a wrong assumption in government that government departments couldn’t talk to the public directly to get feedback on digital products. This was wrong and they started doing UX testing.
  • Create momentum – If things are moving slowly, and they do in government, create your own momentum. Have weekly goals and hit them.

Jeff Gothelf (@jboogie) – Scaling agile, bringing agile to the entire organisation

We have different techniques in different disciplines to try to get a more startup-like culture. Product has lean, tech has agile and design has design thinking. Startups don’t have this problem because they’re startups. People just have to get shit done.

Enterprises do have this problem because there is a relatively infinite runway and less consequences when things do go wrong.

Lean and agile seem to fail at scale. Outside of small teams there isn’t really an appetite for agile or lean. It’s over complicated (see agile landscape v3) and we have complex process definitions like SAFE (http://www.scaledagileframework.com/) which are often taken literally therefore missing the point of the flexibility required for moving like a startup.

The answer is to scale principals NOT processes. Some principals to bring in to the organisation are…

Customer value = Business value

This might seem obvious in startups and smaller teams but it should be explicitly worked on  at scale. OKRs are a god way to provide focus. They should be qualitative, inspirational, temporal and actionable. Recommends a book Radical Focus for reading more about OKRs. Gave example of car dealer site OKR. A tactic for enterprise is to plan only in quarters. Software is too complex for yearly planning. Another tactic is to use horizon planning.

Value learning over delivery

Tactic for prioritizing learning include having pilot teams working in new ways to learn from. Use the best people in the organisation. Provide sandboxes and guard rails for the teams. Encourage creative experiments.

Radical transparency

Have transparency through rituals, through access to information, access to customers, access to business data.

Humility in all things

Keep a beginners mind. “Strong opinions, loosely held”. Do product discovery. Use an orbital model for product development support (1-2 hours / week from marketing, legal etc.). Use a modern staffing model. Do you need all the traditional roles and bureaucracy. Modernize your tech stack. If the tech wont let you scale agile then make it better.

All of this should be a top-down driven effort.

Indy Johar (@indy_johar) – We need to re-imagine our institutional infrastructure for the digital age

Democracy is not just “the vote”, it is a stack distribution of agency

Gave examples of digital era products Domus and Wiki house where the pans for furniture or housing are available online and local companies build and ship them to people. These are innovative ideas but the regulatory stuff around multi jurisdiction house plans doesn’t exist yet. We have ignored the red-tape problem.

When the industrial revolution was underway in Britain the British Standards Institute started along side of it because it was a necessary system. We need to look at the dark matter, the bits in between our digital products. We should focus on new regulatory infrastructure instead of creating new products as a thin layer on top of old regulatory systems.

We have been living in an age with the cult of the individual. Western children when shown a painting will immediately focus on the object. Eastern children will focus on the context and will only focus on object when questioned about it directly.

We focus too much on single causes for our social problems but often there are many causes. The problems are complex. There can be surprising results from non-obvious solutions. Example of a town growing food in every available public space. Depression rates dropped, unemployment dropped. But try saying to the tech community “We don’t need a tech fund, we need to grow cabbages” – people won;t take it seriously. Buying activities vs buying outcomes == depression drugs vs growing cabbages.

Darius Kazemi (@tinysubversions) – Doing tech for activists

  • Use simple stuff
  • Don’t recreate, mutate
  • Technology is terrible, avoid where possible

Genevieve Bell (@feraldata) – Talking to AI

Lot’s of thought provoking ideas presented as a list of questions we would ask AI

AI has lots of daddies but not too many mommies.

The name itself is a sign of its time. In the 50s “artificial” was a good thing. Not such a good thing now days. “inteligence” doesnt suggest much feeling. What would we call it today?

Anil Dash (@anildash) – Humane tech

The internet is moving regulation in to private companies. For the worst.

The modern internet is not an efficient market. Opaque algorithms cannot be regulated.

http://neveragain.tech/

Stefan Sagmeister (@sagmeisterwalsh) – Why beauty does matter in everything

Modernist buildings are horrible to live in. The modernists themselves renounced their lack of beauty. Gave some great examples of how adding beauty to things changes the way people feel and think. Underpass where people used to piss all the time. Tey had $5k to prevent the urinating. Painted “Yes” on each side of the wall and it became a place where newly weds would do photos. From industrial train transport to the skyline park. There has not been a single act of crime on the skyline park.

How if NASA added beauty to their suit it could become a talking point for more funding. People like supporting beautiful things. How Air bnb and Etsy have bland sparse design. It could be more beautiful. Showed some of his own work.

Smart Bear Ready API: Custom headers per request

I recently had a problem where our test analysts use Ready API software from Smart bear but our API authentication does a custom hashing function on the content. For tests to work we had to hash the entire request and apply headers for each test step.

There is an events hook in Ready API where you can add Java code to be run for event triggers. Unwrapping the content in the available request context API was difficult to figure out.

First open the Events dialog. Add a new event for RequestFilter.filterRequest to do something for every test step with a request.

Ready api event entry window

Here is the script.


import java.nio.charset.StandardCharsets;
import java.security.SignatureException;
import java.security.spec.EncodedKeySpec;
import com.eviware.soapui.support.types.StringToStringMap;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.io.UnsupportedEncodingException;   
import java.security.MessageDigest;
import javax.xml.bind.DatatypeConverter;
import java.sql.Date.*;
 
log.info "=============================";
 
// CHANGE THESE
//====================================================================
String apiUsername = "someApiUser";
String mySiteUsername = "readyAPI@nowhere.co.nz";
String secret = "my special secret";
//====================================================================
// DON'T CHANGE BELOW HERE
//====================================================================

// set up some header names we will add to the request
String AUTH_HEADER_NAME = "X-mySite-Authorization";
String TIMESTAMP_HEADER_NAME = "X-mySite-Timestamp";
 
SecretKeySpec signingKey = new SecretKeySpec(secret.getBytes("UTF-8"), "HmacSHA1");
String timestamp = new Date().format("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'", TimeZone.getTimeZone("UTC"));
log.info "TimeZone.getTimeZone(UTC); " + TimeZone.getTimeZone("UTC").toString();
 
String requestResource = context.httpMethod.getURI();
String requestMethod = request.method;

// This is how you get access to the request content!
String requestContent = context.expand(request.requestContent);
log.info 'requestContent: ' + requestContent;

// AWS hashing algo start here
byte[] b = requestContent.getBytes("US-ASCII");
log.info b.toString();
 
String signature="";
String bodyDigest = ""; 
 
try {
// Generate a body digest if there is content in the request
if (requestContent != null && requestContent.length() != 0){
        MessageDigest md = MessageDigest.getInstance("SHA-256");
        md.update(requestContent.getBytes("UTF-8"));
        byte[] digest = md.digest();
        bodyDigest = DatatypeConverter.printBase64Binary(digest);             
}          
 
// Create a concatenated message string to sign
String message = new StringBuilder().append(requestMethod).append(requestResource).append(timestamp).append(bodyDigest).toString();
log.info "Message: " + message;
// Get an instance of an HmacSHA256 hash and hash the message use the signing key
Mac mac = Mac.getInstance("HmacSHA1");
mac.init(signingKey);
 
byte [] signatureBytes = mac.doFinal(message.getBytes(StandardCharsets.UTF_8));
 
signature =  DatatypeConverter.printBase64Binary(signatureBytes);
 
} catch (Exception e) {
    throw new SignatureException("Failed to generate HMAC : " + e.getMessage());
}


//log all the things for debug!
 
log.info "requestMethod: " + requestMethod;
log.info "requestResource: " + requestResource;
log.info "timestamp: " + timestamp;
log.info "requestContent: >" + requestContent + "<";
log.info "apiUsername: " + apiUsername;
log.info "secretKey: " + secret;
log.info "=============================";
log.info "=====     HEADERS     =======";
log.info "=============================";
log.info TIMESTAMP_HEADER_NAME + ": "+ timestamp;
log.info AUTH_HEADER_NAME + ": "+ apiUsername + ":"+signature;
log.info "=============================";

// Write out all the headers to the request context
// and add them to the test step
def headers = new StringToStringMap();
headers.put(AUTH_HEADER_NAME,apiUsername + ":"+signature);
headers.put(TIMESTAMP_HEADER_NAME,timestamp);
 
request.setRequestHeaders(headers);

Setting up a wordpress blog with SSL and custom domain for $6 per month

When I started this blog I wanted to host it on Azure but it would have cost roughly $100/month with my requirements of SSL and custom domain. There are some nice benefits of Azure but I realised I needed a blog and WordPress is the king of blogs.

I started my blog on wordpress.com just so I could start writing before the enthusiasm fizzled out. This is really awesome for free blogs but the costs quickly add up if you want to add a custom domain, SSL and google analytics. All of this requires the business package and costs $25/month (at time of writing). You are also prevented from installing many themes and plugins here. I did some searching for a cheaper host and settled on DigitalOcean .

Create the WordPress droplet

Digital Ocean is a container based host. They call the containers droplets. To create your cheap WordPress site go to DigitalOcean and sign up. You get a month free with that link. After signing up select “Create Droplet”. In the page that opens click the “One Click Apps” tab.

Digital Ocean one click selection

 

Scroll down and select the WordPress 4.7 on 16.04

Digital Ocean wordpress application selection

 

Select the smallest size droplet. This should be fine as you start.

Digital Ocean price selection

 

I recommend getting a backup of your droplet. This is a weekly backup for $1/month. Pretty good deal I’d say.

Digital Ocean backup selection

 

That’s it! Just wait for the droplet to spin up. You’ll get an IP address and username password in an email. The IP is also available on your Droplets page.

Setup the WordPress blog

On windows you’ll have to download putty to log in to your new droplet. Visit http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html and get the full msi installer.

Putty msi file

Use the IP, userame and password to log in to the new droplet using putty.



<pre class="code-pre "><code>ssh root@192.168.1.1</code></pre>

Next visit your wordpress site in your browser – http://192.168.1.1 – but replace the IP there with the one for your droplet and follow all the prompts to install WordPress. Don’t use “admin” as the main account.

Setup some security

Next you should set up some security on the droplet. run the following to display your passwords. The mysql secure installation script will ask for the root password later.


cat /root/.digitalocean_password

Then run

 mysql_secure_installation 

and follow the prompts. You probably want to say yes to everything.

Now make sure mysql is started

 service mysql start 

Setup your domain

If you have a custom domain then digital ocean can act as a nameserver for you for free. It’s best to configure digital ocean first before changing nameservers on the domain. If this is a new domain then add the following records under networking.

Create TXT records for


www.mycustomdomain.com VALUE www.mycustomdomain.com

*.mycustomdomain.com VALUE www.mycustomdomain.com

mycustomdomain.com VALUE www.mycustomdomain.com

and A records for


www.mycustomdomain.com VALUE 192.168.116.115

*.mycustomdomain.com VALUE 192.168.116.115

mycustomdomain.com VALUE 192.168.116.115

Now change the nameservers for your domain to point to digital oceans name servers.

Set up SSL

You can get free SSL with Let’s Encrypt. First get cert-bot.


sudo wget https://dl.eff.org/certbot-auto

then make it executable.

sudo chmod a+x /usr/local/sbin/certbot-auto

Then run for the domains you would like to add certs for.

 certbot-auto --apache -d mycustomdomain.com -d www.mycustodomain.com 

Using a cron job to renew your certs automatically

Open your crontab file

sudo crontab -e

and add the following line to the end of the file but replace XX with a number between 0 and 59

 XX 2 * * 1 /usr/local/sbin/certbot-auto renew >> /var/log/le-renew.log 

Start Writing

That’s it! You’re done. A wordpress blog with SSL and custom domain for $6.