hennevogel.de

Let's build Software Libre APM together

Fri, 26 Mar 2021 00:00:00 +0000

Alright, you learned about ActiveSupport::Notifications, InfluxDB, Grafana and influxdb-rails in the two previous posts. Let's dive a bit deeper and look how we built the dashboards for you. So we can study, change and improve them together.

“Individually we are one drop; but together we are an ocean.” – Ryunosoke Satoro

Welcome to your Ruby on Rails Application Monitoring 101.

This post is the last part of a three part series about monitoring your Ruby on Rails app with influxdb-rails. Make sure to check them all out!

ActiveSupport::Notifications is rad! 💘
Measure twice, cut once: App Performance Monitoring with influxdb-rails
Let's build Software Libre APM together (this post)

The Ying and Yang of measurements

Basically there are two types of measurements we do. How often something happened and how long something took. Both types are most often complementary, interconnected and interdependent. On the performance dashboard we count for instance how many requests your application is serving.

We also look at the time your application spends on doing that.

At some point the number of requests will have an influence on the time spend (oversaturation). If one of your actions is using too many resources it will have influence on the number of requests you are able to serve (overutilization). Ying and Yang.

The minions of measurements

Looking at the graphs above you see some helpers at work you should know about.

Time

First and foremost: Time Windows. The requests are counted per minute and we look at measurements in the last hour. Time windows help to lower information density and make your measurements a bit more digestible. Look at the same measurements per second for the last 12 hours. 60 times 24 higher information density. Not so easy to interpret anymore...

Statistics

The same way time windows will make it easier for you to understand your data, some descriptive statistics will help you too. For instance calculating the maximum time 99% of requests in the last minute took. Like we did above. I won't bore you to death with the math behind this, if you're into this checkout wikipedia or something. Just remember, it makes data digestible for you. Look at the same performance data without applying statistics.

Grouping

The third helper that makes data understandable for you is grouping data. Like ActiveJobs per minute grouped by queue. That grouping might make more clear to you why the number of jobs so high. Or grouping requests per minute by HTTP Status might reveal how much stuff goes wrong.

A different form of grouping is to visualize all measurements connected to a specific event, like a single request. Or all the measurements a specific controller action has fired in the last hour.

Rankings

Another thing we do on the dashboard is ranking (groups of) events by time, slow to fast. So you know where you might want to concentrate your efforts to improve performance.

Apply your knowledge, let's collaborate on Rails APM/AHM

Maybe I inspired some ideas for new features in the 101 above? I'm also sure there are many people out in the Rails community that have way more knowledge and ideas about statistics, measurements and all the tools involved than Chris and me. Let's work together, patches to the collection of dashboards (and the Ruby code) are more than welcome!

Why build, not buy?

But Henne, you say, there is already Sentry, New Relic, Datadog, Skylight and tons of other services that do this. Why build another one? Why reinvent the wheel?

Because Software Libre is an deeply evolutionary process. Software Libre, just like Evolution, experiments all the time. Many experiments find their niche to exist. Some even go global.

Coronavirus: BW CG Illustration by Yuri Samoilov

Like Linux, the largest install base of ALL operating systems on this planet. Wordpress powering an unbelievable 30% of the top 10 million websites. Mediawiki running the 5th most popular site globally. But also, more than 98% of all projects on GitHub are not seeing any development beyond the first year they were created. Just like 99% of all species that ever lived on Earth are estimated to be extinct.

We need to experiment and collaborate together. Evolution is what we do baby! 🤓 Can't run, copy, distribute, study, change and improve the software SaaS providers run. We can't scratch our itch, can't break it, learn how it works and make it better together, that's why. Let's do this!

Measure twice, cut once: App Performance Monitoring with influxdb-rails

Thu, 25 Mar 2021 00:00:00 +0000

Now that we learned how truly magnificent ActiveSupport::Notifications is in the previous post, let's explore a RubyGem Chris, others and me have built around this: influxdb-rails. Together with even more awesome Software Libre, it will help you to deep dive into your Ruby on Rails application performance.

This post ist part two of a three part series about monitoring your Ruby on Rails app with influxdb-rails. Make sure to check them all out!

ActiveSupport::Notifications is rad! 💘
Measure twice, cut once: App Performance Monitoring with influxdb-rails (this post)
Let's build Software Libre APM together

Application Performance Monitoring (APM)

I'm not sure we have to discuss this at all anymore, but here is why I think I need application performance monitoring for my Ruby on Rail apps: I'm an expert in using the software I hack. Hence I always travel the happy path, cleverly and unconsciously avoid the pitfalls, use it with forethought of the underlying architecture. Because of that, I usually think my app is fast: perceived performance.

It's not what you look at that matters, it's what you see. – Henry David Thoreau

That is why I need someone to correct that bias for me, in black and white.

Free Software APM

The good folks at Rails bring the instrumentation framework. InfluxData deliver a time series database. Grafana Labs make a dashboard builder. All we need to do, as so often, is to plug Software Libre together: SUCCE$$!

InfluxDB + Grafana == 🧨

I assume your Rails development environment is already running on 127.0.0.0:300. Getting InfluxDB and Grafana is a matter of pulling containers these days. You can use this simple docker-compose configuration for running them locally.

# docker-compose.yml
version: "3.7"
services:
  influx:
    image: influxdb:1.7
    environment:
      - INFLUXDB_DB=rails
      - INFLUXDB_USER=root
      - INFLUXDB_USER_PASSWORD=root
      - INFLUXDB_READ_USER=grafana
      - INFLUXDB_READ_USER_PASSWORD=grafana
    volumes:
      - influxdata:/var/lib/influxdb
    ports:
      - 8086:8086
  grafana:
    image: grafana/grafana:7.4.5
    ports:
      - 4000:3000
    depends_on:
      - influx
    volumes:
      - grafanadata:/var/lib/grafana
volumes:
  influxdata:
  grafanadata:

A courageous docker-compose up will boot things and you can access Grafana at http://127.0.0.1:4000 (user: admin / password: admin). To read data from the InfluxDB container in Grafana, leave the /datasources InfluxDB defaults alone and configure:

URL: http://influx:8086
Database: rails
User: grafana
Password: grafana

influxdb-rails: 🪢 things together

The influxdb-rails RubyGem is the missing glue code for making your app report metrics into the InfluxDB. Plug it into your Rails app by adding it to your bundle.

bundle add influxdb-rails
bundle exec rails generate influxdb

The next time you boot your Rails dev-env it will start to measure the performance of your app. Now comes the interesting part, interpreting this data with Grafana.

Understanding Ruby on Rails Performance

Every time you use your dev-env, influxdb-rails will write a plethora of performance data into InfluxDB. Let's look at one of the measurements so you get an idea of what's going on. You remember the ActiveSupport::Notification called process_action.action_controller from the previous post? Rails sends this message every time an action in your controller has finished. It includes performance data for this action.

You should know this from somewhere: development.log! It contains the same information.

Started GET "/things" for 127.0.0.1 at 2021-03-25 15:20:14 +0100
...
Completed 200 OK in 5ms (Views: 4.1ms | ActiveRecord: 0.1ms | Allocations: 3514)

ThingsController#index took 5ms to finish overall, 4.1ms of those 5 in rendering, 0.1ms in querying the database. You find the same data for every request you make in your InfluxDB. Head over to http://127.0.0.1:4000/explore and let Grafana plot it for you.

Only want to see how your views are performing? Change the field from controller to view. Magic 🪄 But this is only one out of many different ways to look at this measurement. All of the panels below use this one measurement and the data it brings to visualize controller actions.

And this is only one measurement, influxdb-rails reports around a dozen. Now I could send you off your way to learn about ALL. THE. SOFTWARE. involved, but that would be mean wouldn't it?

Ruby on Rails Dashboards

We, the Free Software community, are in this together! We collaborate on Ruby on Rails. We work together to make InfluxDB better. We cooperate to improve Grafana. Why not do the same for the dashboards to visualize Rails performance data? Let's collaborate! That is why we have build a couple of dashboards you can import. Just copy and paste the URLs into your Grafana.

Ruby On Rails Performance Overview: Ruby On Rails Performance Overview
Performance insights into individual requests: Ruby On Rails Performance per Request
Performance of individual actions: Ruby On Rails Performance per Action
HTTP Request Health: Ruby On Rails Health Overview
ActiveJob Insights: Ruby on Rails ActiveJob Overview
A list of the slowest requests: Ruby on Rails Slowlog by Request
A list of the slowest actions: Ruby on Rails Slowlog by Action
A list of the slowest queries: Ruby on Rails Slowlog by SQL

Play a little, I will tell you about all the nitty gritty details in the last post of this series: Let's build Software Libre APM together

ActiveSupport::Notifications is Rad!

Wed, 24 Mar 2021 00:00:00 +0000

One of the lesser known parts of Rails core is the ActiveSupport instrumentation framework. ActiveSupport::Notifications includes all the things you need to implement pub-sub in Rails. Pub-Sub is a software architecture where you publish (send) a message without being specific about who should receive it. Fire and forget.

Receiving a message, and doing something with it, "just" requires you to subscribe to it. Because the publisher doesn't need to know about the subscribers (as they are decoupled), this provides nice opportunities for organization and scale.

Let's explore the joyful shenanigans of this.

This post is the first part of a three part series about instrumenting your Ruby on Rails app with influxdb-rails. Make sure to check them all out!

ActiveSupport::Notifications is rad! (this post)
Measure twice, cut once: App Performance Monitoring with influxdb-rails
Let's build Software Libre APM together

Publish & Subscribe

There is an instrumentation message emitted from ActionController that includes interesting data about the things happening in your controller action. Let's explore this.

If you don't have a Ruby on Rails app at hand, just setup a minimal one with rails new --minimal

Add this code into an initializer

# config/initializers/instrumentation_subscriber.rb

# What happens in ActionController?
module ActionController
  class InstrumentationSubscriber < ActiveSupport::Subscriber
    attach_to :action_controller

    def process_action(message)
      Rails.logger.debug "Instrumentation #{message.name} Received!"
    end
  end
end

Boot the app (rails server), visit http://127.0.0.0:3000 and you'll see the new log lines in your development.log. So what? What's the difference to calling Rails.logger in an action or callback inside your controller? Why is ActiveSupport::Notifications fabulous?

ActiveSupport::Notifications Scales

First, as explained in the intro, the main advantage is that the publisher is decoupled from the subscriber. For instance, you can have more than one subscriber listening to process_action.action_controller.

# config/initializers/slowlog_subscriber.rb

module ActionController
  class SlowlogSubscriber < ActiveSupport::Subscriber
    attach_to :action_controller

    def process_action(message)
      return if message.duration <= 10

      controller_location = [message.payload[:controller], message.payload[:action]].join("#") # -> ThingsController#something
      Rails.logger.debug "#{controller_location} was slow (#{message.duration}ms)" 
    end
  end
end

You are free to organize this however you want. Decouple publisher/subscriber in different files, chronological or even in different threads.

# config/initializers/poor_mans_background_job_subscriber.rb

module ActionController
  class PMBGJSubscriber < ActiveSupport::Subscriber
    include Concurrent::Async

    attach_to :action_controller

    def process_action(message)
      async.background_job(message)
    end

    def background_job(message)
      # ...do something expensive with the message in a thread
      sleep(60)
    end
  end
end

Activesupport::Notifications Is Everywhere Today

Second, and you probably already guessed this from the example, what makes this most awesome are the ready made messages that are already published today.

Rails for instance uses ActiveSupport::Notifications to publish close to 50(!) different instrumentation events that include data about your application. Data ranging from the HTTP status code of your requests, over which partials were rendered, to more esoteric measurements like the byte range attempted to be read from your ActiveStorage service. Check the instrumentation guide for all the dirty secrets.

ActiveSupport::Notifications Is Easily Extendible

Last but not least, you can not only listen to messages others publish, you can publish messages to yourself.

ActiveSupport::Notifications.instrument "cool_thing_happened.my_app", {some: :data} do
  MyApp.do_cool_thing(true)
end

Okay, you're convinced I hope! Now what do people do with ActiveSupport::Notifications out in the world?

Application Health/Performance Monitoring

Ever wondered how your metrics get to Sentry, New Relic, Datadog or Skylight? You guessed it, via ActiveSupport::Notifications.

Now if the main work, publishing tons of messages through a pub-sub architecture, is already done by Rails, surely there are non-SaaS (a.k.a. Software Libre) options to display this data, right? Turns out there are not. While evaluating options for some of my projects (especially Open Build Service) we came across this gap and started to fill it.

How, why and where? Read on in the next part of this series: Measure twice, cut once: App Performance Monitoring with influxdb-rails

My Toolbelt

Mon, 02 Mar 2020 00:00:00 +0000

Here is my personal hardware & software stack. The things I use and enjoy on a daily basis since a long time. The software is pretty much the standard in Free Software development stack. Don't expect to find anything you don't know here. I write this because people keep asking me, especially in mentoring situations.

Operating System: openSUSE Tumbleweed

It's my Linux of choice. Why? Because it's the German Engineering version of Linux: Long heritage, a certain kind of stiffness and it's super tidy. The rolling nature of Tumbleweed gives me the stability and the velocity I need. It brings all the standard Linux tools through a kick ass package manager (zypper). And it embodies what makes the Free Software eco system great: software diversity. openSUSE tries very hard to treat all software as first class citizen of the distribution.

Okay I admit it, I have been involved in making this Linux distribution for decades. So take this as pure bragging on my part. What it also means for me is that it's super easy to "scratch my itch" a.k.a. understand things, fix bugs, implement features and find the right people to poke. I can only recommend this to you, find a Linux home, help to maintain it and make it your own.

Desktop Environment: GNOME

The thing I like most about GNOME is that it get's out of my way. It's restraint, simplistic, straight forward. And yet, if I want to do a thing I have never done before, it'll have an opinion on how to do it. I also dig the esthetics, I think I actually switched to it because it's soo damn beautiful to look at.

GNOME 3 Desktop by gnome.org

Shell: zsh

Well, any shell. I couldn't live without all those glorious one-liners like for i in *; do echo mv $i `echo $i |sed 's/.rb/_spec.rb/'`; done or history | awk '{print $2}' | sort | uniq -c | sort -nr |head -n 20. Utilizing all those programs that do only one thing, but one thing well. cp, mv, echo, grep, cat, awk, sed, find, head, tail, curl, test, sleep and so on and so on. What a wonderful chest of lego bricks, ready for you to plug together your own world. Also, I literally grew up in bash. It was shell scripting that introduced me to software development. And I will forever be fascinated by it, despite all the sneeze from the more sophisticated languages. Shell scripting is messy, raw, dangerous. Shell scripting is beautifully punk!

Browser: Mozilla Firefox

I think, not counting the Linux kernel, Firefox is the only piece of software I used all my life (if you think of netscape communicator as ancestor). It's because the single most important quality of Firefox is progress. Tabs, add-ons, session restore, firebug, sync, mobile version and now built in privacy. Mozilla has been steadily pushing it's flagship product forward. It might not always be the first to market, it might not always be perfect from the start but it is always moving forward relentlessly. That is a rare quality. We all should pray that Mozilla continues this despite the google browser monopoly. BTW how the flying f**k is that monopoly OK with 65% of you? Don't you remember any computing history? </rant>

Source Code Management: git

In the meanwhile all projects I contribute to use git. I like git for being the right amount of clever CLI. That opens the pager only in the right moment, corrects me if I typo or let's me extend it, just by nameing a command git-diff-to-deploy. And I really dig the file based approach which means that I can often recover my idiot mistakes by nervously mocking around in .git. It shows that the people maintaining this are seriously dog-fooding their own product day in and day out.

Source Code Collaboration: github

With github I have a love-hate relationship. I admire the idea, the product and many of the people who build it. I build a tool in the same space of the industry (developer tooling) using the same technology (Ruby on Rails) and github people set the gold standard in collaboration workflows since years. I only humbly follow their set out examples.

But I really don't like them riding and emphasizing the network effect. And I flat out hate that such a large piece of software in my stack is closed source. Quite frankly I do not understand how anyone at Microsoft can justify to their mirror that github is not Free Software. It's not like anyone will ever be able to take any business away from github.com which value is the number of people using it. As you can see with me and my projects, that ship has sailed a long time ago. Why do you have to deny me my software freedoms if I can't use them to hurt you in any way?

I am a Ruby on Rails developer, I know a thing or two about collaborating on software. I want to send a pull-requests fixing gripes or missing functionality I have with github itself. Why the hell can't I do that? Why do people have to resort to write browser extensions to be able to collaborate on github functionality. This is insane! </rant>

Mailer: Mozilla Thunderbird

Thunderbird is the only piece of software in this list I'm not very passionate about. It helps me reading, finding and writing mail from my four imap accounts in one single interface. It does so better than any mail user agent, web mailer or mobile app I have tried so far. Yet I have the feeling there should be something better. Can't seem to find it though, despite really searching. Guess Churchill was right: Thunderbird is the worst MUA, except for all the others.

Editor: Visual Studio Code and vim

I fire up vim when I need to do a quick script, jog down a note, edit configuration files or for a quick fix. Visual Studio Code is what I use for deep dive bug fixes or feature development. About vim I like most the raw power that comes with it. Doing something like gqaw or v<end>~ is gold. But I have a problem with vim: I easily remember a small set of things for a long time, but I have not much capacity to remember 50 keyboad shortcuts forever ¯\_(ツ)_/¯ That's why I need a user interface that guides me and offers me a place to search visually for what I want to do. CTRL+p+fold level+click is exactly that.

Visual Studio Code by Henne Vogelsang licensed CC BY 4.0

Containers: docker & docker-compose

I'm hacking a bunch of different software projects on a daily basis. I need them compartmentalized so I can build, start/stop and destroy them at will. Without each of them compromising the other, or my system, with their gazillion dependencies, services and side cars.

Linux containers are the most efficient way to do this for me. As I'm not super interested in architectural intricacies of container runtimes and a frequent visitor of doineedkubernetes.com, I'm kind of habituated to docker and docker-compose. Investing a day or two into building a development environment that you can set up, run, suspend and tear down at will is hacker magic! Mounting your local file system into it, so your can shell around and use whatever editor you want is 🤯

Laptop: Dell XPS 13 (9380)

I work from at least two different places during the day. My office, meeting room, home office etc. That's why I dig this powerful 13" laptop you can swing around with ease. It also features a nice enough keyboard, can be charged via USB-C (power bank ready) and you can even be buy it with (the wrong) Linux preloaded! What more can you ask from hardware?

Keyboard: Ducky One 2 Horizon TKL

Guess if you're an old fart like me you just need something that feels a bit like a model M keyboard. But please without the nicotine stained look. This ducky quacks for me at the office where I plug in my laptop into a docking station.

Ducky One 2 Horizon TKL duckychannel.com.tw

That's what I consistently make use of during the day, nothing spectacular right? Of course not, those things are about getting shit done! Not about impressing you, sport! 😜

Fascinating! February 2020

Sun, 01 Mar 2020 00:00:00 +0000

The universe is full of captivating, compelling and exquisite things. The internet, the most complete representation of humanity, is no different in that regard. Both are vast, seemingly endless and full of places you should know about. Let me share content that has moved me, in some way, in the last month. February was mostly inter-personal stuff like ethics, hiring or collaborating remote. Enjoy.

YubiKey 5C Nano a product by yubico

Some collegue of mine mentioned andOTP and good Free Software citizen I am, moved all of my 2FA to it. Doing so made me think of checking support for hardware security modules on Linux again. Ordered this YubiKey, now I'm off 2FA with my phone and just touch my laptop when asked. Thanks WebAuthn! One of those "Why they heck did I not do this earlier?" moments.

A student guide for navigating ethical issues in the tech industry by Mozilla

I have no idea why Mozilla says this is for students? Clearly this is for everybody! Be it people considering going into the tech industry (like students), people already in it or even people who just want to know what it going on in this cesspool.

Especially if you are in this industry already: Read this, think about it, talk about it with your peers, dive deeper into this topic. Do this so you can help to save the environment you probably planed to work the rest of your life in. Ethics help them who help themselves!

git sparse-checkout a post by Derrick Stolee

An experimental git command that allows you to checkout one or more sub-directories of a repo. Primarily meant for monorepos with gazillion of projects in it. But I think this could also turn out to be usefull for other situations. Like skipping large infrequently used directories etc. Check it out!

The 2020 State of Remote Work a survey by Buffer.com

No real shockers this year, the concept of distributed teams is humming along. One thing I find interesting about this is how much the top challenges are also the top challenges of (larger) Free Software communities. If you reach a certain size your community fractures because people start their own communication neighborhoods. Mailing lists, forums, IRC etc. Each group get's enough information about the community (mostly from wanderers between the neighborhoods) so they feel they don't miss out. Yet, some important things fall through the cracks. Frequent cries for unification of groups are either ignored or end up in XKCD#927. In the end, you just live with it and realize that you can't streamline a large amount of people to this degree.

And I think loneliness is also at the root of many of the problems that Free Software communities face. Especially I suspect this to be the top reason for contributor churn. You can't vent at the time you're angry about something in the code base. You don't have anyone to share your joy with, at the time you finally feel confident enough to open this super cool feature pull request. You have to context switch to something else because no one is around at the time you need to know why this code behaves like it behaves. Those situations are just a little less fulfilling than they are if you're together with people. And this will bite at your willingness to continue, one at a time. And that's how you can eat an elephant.

Often I'd wish we would not have been such punks in the Free Software movement. That we instead collectively focused some of the time and energy we've spend on creating software on figuring out what happened to us and between us. We made collaboration among free equals into a viable option to be creative. But we were to lazy to write the documentation how to organize people doing it...

The Horrifically Dystopian World of Software Engineering Interviews a post by Jared Nelsen

One persons odyssey through many of the ridicolous hiring practices in play today. And the best analysis of why this is like it is I have read to today.

We are being consumed by our own mythology. We all know algorithm challenges are a contrived game. Why are we measuring people and not learning about people?

I start to feel bad for the nail, as much as this hits it on the head. Or as Chelsea puts it “Smart” is Not a Hiring Criterion.

Tools I found interesting

flameshot · A screenshot tool
scc · Fastest LOC counter possible
Browser Lazy Load · Coming to a browser near you <img src="image.png" loading="lazy">
Git Command Explorer · Interactive git documentation
explorabl.es · A hub for learning through play!
hackertyper · A neat way to show non-nerds what you do for a living
capycorder · Generating capybara request specs by clicking in your browser

Talks I enjoyed

Compassionate—Yet Candid—Code Reviews by April Wensel
Salary Negotiation Workshop by Josh Doody
Implementing a Strong Code-Review Culture by Derek Prior
Single-Vendor Open Source Firms by Prof. Dr. Dirk Riehle
Vom Ich zum Wir by maha and Kai Biermann

Neat stuff! Hope some of it makes you think/act in new ways, as it has me.

Fascinating! January 2020

Sat, 01 Feb 2020 00:00:00 +0000

The universe is full of captivating, compelling and exquisite things. The internet, the most complete representation of humanity, is no different in that regard. Both are vast, seemingly endless and full of places you should know about. Let me share content that has moved me, in some way, in the last month. January is packed with sustainable open source, management stuff, too many tools and a bunch of awesome talks from 36c3. Go explore!

Open source licenses: 2019 year in review a post by Luis Villa

Luis, the co-founder of tidelift, wrote a nice rundown of all the Free Software license developments of the last 12 months. From pushing worker solidarity (996.ICU) over the ethical source movement and the hippocratic license to the rough time the OSI and the FSF have to respond to these new ideas. In 2019 the question that is currently dividing so many people on this planet has arrived (again) in Free Software. What is more important: Safety or Privilege?

XS:CODE a company.

While we are on the topic of licensing. I came across XS:CODE, the third player (I know of) in the realm of subscription based open source. Tidelift and Lizense Zero are the other two. This is still a bit too much gig-economy for me personally, but for sure a nice alternative to producing Free Software as a byproduct of the next big closed source SAAS app. I keep wondering, if something like this would have existed in my (hacker) youth, what would I have done? Probably stayed away from hacking because of choice fatigue…

Building a First Team Mindset a post by Jason Wong

This one hit close to home. As someone who grew out of the world of Free Software into the world of people/project management I'm often amazed on how anti-collaboration this field is. Most managers care a great deal about the team they lead and give a flying fuck about the managers they serve with. All this clan mentality, self-absorption and bickering about resources. Jason proposed to adopt a different approach to this.

A First Team mindset is the idea that leaders prioritize supporting their fellow leaders over supporting their direct reports

Amen Jason!

Hemmingway an app by Adam & Ben Long

Adam and his brother Ben Long have created the Hemingway App to fight the awful prose on the internet. Kind of like... my prose. I have the dreadful habbit to write super long, riddled with subordinate clauses, and superfluous complicated sentences. There, you see? I've been using the app more and more, especially while writing for a broader audience. Like job descriptions, commit messages and blog posts. It's a form of accessibility, the peepz from a list apart agree and 24a11y agree.

Face of Open Source: New Faces added in 2019 portraits by Peter Adams

Free Software moves me most, if it mixes with other art forms. Be it design, music, performance or like in this case: photography.

Faces of Open Source / Peter Adams CC BY-NC-SA 4.0

Look at those beautiful people and check out their contributions.

Culture is the Behavior You Reward and Punish a post by Jocelyn Goldfein

The story of how Jocelyn learned that culture needs to be defended. I really liked the question: "What makes people successful here?" as a tool to understand the lived (not stated) values of an organization.

In the last couple of years lots of projects (especially events) in the software community have been working on defining and defending their cultural standards. Out if this emerged the contributor covenant as defacto standard. It clearly emphasizes the need for enforcement.

A code of conduct that isn’t (or can’t be) enforced is worse than no code of conduct at all: it sends the message that the values in the code of conduct aren’t actually important or respected in your community. — Ada Initiative

At work we are undergoing a phase of similar culture turmoil right now. Not sure all people have understood yet what they got in to.

Our Data Our Selves a project by the tactical tech NGO.

A very deep look at the increasing role data plays in our lives. As means to activism, force in politics and integral part of yourself.

How to Estimate Feature Development Time: Maybe Don't a post by Amanda Beiner of thoughtbot

A very profound argument of the benefits but especially the consequences of (time) estimations in software development. At work we hopped the #noestimates train a long time ago, never really looked back. I personally also begin to dread the same consequences (explicit deadlines) for the sprint SCRUM event. I dream of trying to adopt our iteration cycle to the deliverable, not the other way around.

Tools I found interesting

remark.js · A markdown linter
brow.sh · A modern text-based browser
bandwhich · A terminal bandwidth utilization tool
autocrypt · Convenient End-to-End Encryption for E-Mail
suse.ai · An Open Source personal assistant / chat bot
ffsend · A command line Firefox Send client
db · A database snapshot tool

Talks I enjoyed

That was a lot of awesome stuff! Hope some of it makes you think/act as it has me.

Fascinating! September 2019

Mon, 30 Sep 2019 00:00:00 +0000

The universe is full of captivating, compelling and exquisite things. The internet, the most complete representation of humanity, is no different in that regard. Both are vast, seemingly endless and full of places you should know about. Let me share content that has moved me, in some way, in the last month. This time it's mostly think-pieces, an event and a couple of tools. Enjoy!

The Evolution of Trust by Nicky Case

A lovely explanation of trust in game theory. It's an interactive "game" in which you can try out how the rules of the game and the strategy influence the distribution of archetypes of player. I don't think realy humans act anywhere near as rational and consistent as players in game theory, yet, some effects are incredible to whitness first hand.

Thinking, Shallow And Deep by Thomas Oppong

My zodiac sign is sloth and easygoing is two cornerstones of my doctrine. So I'm not very fond of self-optimization, especially not in the startup context. But somehow this piece by Thomas hit it with me. In it he pulls together a couple of thoughts about the act of deep thinking. How that helps you to avoid confirmation bias, how that is a good thing in the information age of life long learning. But also how current society (unintentionally?) distracts you from it. A nice thread thought, asking powerful questions.

Why Don’t We Just Call Agile What It Is: Feminist by Hanna Thomas

There, she said it! Collaborative, supportive, non-hierarchical relationships that empower people and embrace change are at the core of many progressive feminist theories. And obviously they have been a big influence on agile. An Agile mindset is a progressive feminist mindset. I couldn't agree more.

Incredible Doom (Season Finale) by Matthew Bogart

Ah the '90s! Everything was possible and doomed, at the same time. If you, like me, grew up during this decade then this is for you. A comic series about '90s kids making bad decisions over the early internet. If you grew up some other time, check it out to understand what this internet thing really meant in the beginning. Might explain why so many of us still won't let go of the quirky edges of it, despite all the surveilance capitalist bullshit on the frontpage now. Especially the printed issues make me drool.

PROVIDED AS IS by the people at Formidable

Another event targeted at free software/open source project maintainers with the expressed intent to explore the set of skills beyond “just coding”. At some point in time some of us are going to have to pull all of this shared knowledge about tactics out of conversations into a book or something.

indexter by Chris Cummer

Ruby gem for checking Rails database tables for missing indexes on foreign keys. Happy hunting, hope you find at least one that has a nice performance impact on your app.

Redash & Metabase

I'm on an ongoing quest for the perfect Free Software observability sidecar. I need oberservability. I dread every day I wake up to Mysql2::Error::ConnectionError for some requests in my exception tracker without the ability to ask it about the status of the network or SQL cluster during that time. DIGAME! Both tools focus on sharing questions people come up with, as collaboration addict I very much like that. Both are Free Software with people making money from it, like that even more!

That's it, hope you enjoy this stuff as much as I did.

Free Software is funded by Free People

Tue, 31 May 2016 00:00:00 +0000

More and more security incidents in Free Software projects get publicized in a way that anyone & their grandma understand the impact. Some of these carefuly crafted "campaigns" even make it to the main stream media. This trend, among other things, has sparked a discussion about funding Free Software to sustain the infrastructure everyone is building upon. A discussion which, in our venture capital driven tech world, mostly evolves around financing as a means for getting resources. Who can/should pay whom, to develop Free Software and how. I would like to add another view at funding, one that in my opinion has way more to do with Free Software ecosystem than cold, hard cash: collaboration of free people.

Freedom and what you make of it

The ability to run, copy, distribute, study, change and improve software. This is the essence of it, without these fundamental rights, software is not Free Software. And these fundamentals imply a purpose: Freedom, the ability to do as one wills. To do to software as one wills.

As always, what people make of their freedom is up to them. Some go an preach about it, most ignore it, some use it for their personal gain. However, it's undeniable that the Free Software movement has entailed a community of systematic world wide collaboration. In software development, and I would argue in many other aspects of todays life. We all know the examples of hundreds of developers hacking together on Linux, thousands of authors on Wikipedia per month, millions of pictures in the creative commons and so on. Obviously, if you give people the freedom to do as they want, many of them are inspired to work together for the common benefit. And because Free Software entails collaboration and collaboration is about individuals, making projects sustainable has many dimensions. Money is one.

Challenges and what you do about them

Of course there are many things going wrong today with making Free Software projects sustainable. For instance a growing imbalance between developers and users (Noah Kantrowitz), that developers think they are not getting fairly compensated (Ryan Bigg) or that everybody is waiting for someone else to do something (Marijn Haverbeke). Contributor burn-out. There are many, many more.

The solutions to these shortcomings, discussed in the past months, most exclusively focus on monetizing work. Be it through business models, crowd funding, grants and other forms of injecting money into the equasion. In a recent piece, Nadia Eghbal even argues that somehow venture capital must flow back because all of the VC funded software companies somehow base their work on Free Software. Steve Klabnik thinks this is already happening. Here is the thing: Money will buy you people who contribute, sure thing. But is it money for contributions that made the present? For sure not. Collaborative contributions are the currency of Free Software.

openSUSE Community by Henne Vogelsang licensed CC BY 4.0

People and how you treat them

People have made Free Software as big as it is. Active contributor communities. Humans from all over the world, with different talents, levels of expertise and cultural backgrounds collaborating. All with their own motivations. Some want to learn from others, some scratch their itch, some want to train their skills, some want to boost their resume, some contribute on behalf of a company, some just try to avoid being bored on a Saturday afternoon. If your project does not actively channel all of these ambitions into contributions, if it does not organize the collaboration, if it is not offering shared responsibility and if it does not attract new contributors it's doing something wrong.

An individual/company slapping a license onto code, uploading it to github, merging pull-requests and answering issues is not making a sustainable Free Software project. Yet this is what many do today.

If you look at the most prominent examples of the software people worry about, openssl or NTP, they are failed communities. They disintegrated to a point where collaboration ceased to exist, the remainaing people were not able to do anything to change this (I'm sure they tried hard) and in the end something bad happened, and now they are clearly not sustained. Now we all can throw some bucks together to keep developers developing but that wont make those projects sustainable Free Software communities. In my opinion paying for contributions is to capitulate in the face of to the task of building a sustainable community. There are other things which are more important.

Fund Free Software without money

I guess the most obvious way to fund Free Software projects without money is to contribute in any way possible for you, duuhh. But apart from that I think there are more things we as the Free Software community at large can do better.

In my opinion it's time we help each other to organize collaboration. We need to share best pratices about creating sustainable contributor communities that attract new people.

We also should be more upfront with each other when we notice communities failing. I have contributed code to NTP a couple of years ago and didn't care that it was such a bleak place. Someone merged my patches, I was off again. In retrospecitve this feels like not telling someone their flap is open.

I also think that we need to do better with copyleft project resources. We have the choice between 27 different C++ compilers but if you're looking for marketing materials, web site themes or templates of contributor guidelines you will draw mostly blanks. Convincing prospective users and contributors that our Free Software does something significantly better than the competition is very important, yet we leave each other hanging with this.

Now go do something

Luckily somewhere Free Software people already have started with all of this. There is the community leadership forum where you can talk about all the aspects of your community. opensource.com curates a blog of best practices. Designers started to rally around opensourcedesign.net. People all over start to address the obvious diversity problem our community has. Next month (July 2016) there is a conference to talk with other Free Software people about Open Source & Feelings and I'm sure there is more going on out there.

What do you think about financing Free Software projects? Are you going to do something about making Free Software sustainable?

Yours truly, Henne 💚

Mind the gap between platform and app

Wed, 04 May 2016 00:00:00 +0000

With the Open Source Event Manager (OSEM), one of the Ruby on Rails apps I hack on, we're heading down the road to version 1.0. A feature we absolutely wanted to have, before making this step, was an easy deployment to at least one of the many Platform as a Service (PaaS) providers. We deemed this important for two reasons:

Before people commit to use your precious little app they want to try it. And this has to be hassle free.
Getting your server operating system ready to deploy Ruby on Rails applications can be tedious, too tedious for some people.

So I have been working on making our app ready for Heroku which is currently the most popular PaaS provider for rails (is it?). This was an interesting road, this is my travelogue.

Storage in the public cloud

Storing files is incredibly easy in Rails, there are many ready made solutions for it like paperclip, dragonfly or carrierwave. The challenge with Heroku is, that on their free plan, your apps storage will be discarded the moment it is stopped or restarted. This happens for instance every time during a deployment or when the app goes to sleep because nobody is using it.

And even though it's easy to store files in Rails, we in the OSEM team have long discouraged this in our app. We rather try to make it as easy as possible to reference things you have shared somewhere else. Want to show a picture of your Events location? Use the ones you share on flickr or instagram anyway. Embed a video of a Talk? Just paste the Youtube or Vimeo link. Share the slides with your audience? Slideshare or Speakerdeck to the rescue!

OSEM commercials by Henne Vogelsang licensed CC BY 4.0

Still there are some places left in our app where we upload pictures. Pictures we think conference organizers are not necessarily free to share on other platforms, like sponsor logos or pictures of sponsored lodgings. So to be able to use OSEM on Heroku our file upload needed support for offloading the files to someone else's computer a.k.a. *the cloud*. In the end I have settled for the carrierwave plug in of cloudinary (pull request #970).

This means it's now as easy as configuring the cloudinary gem and making use of their free plan to shove off all the storage OSEM needs to them.

Storage was the first gap in OSEM that I closed, another piece of the puzzle was configuration.

Configuration in the environment

According to some clever people it's too easy to mistakenly check in your apps configuration file into your version control system. That's the reason your apps environment should provide all the settings. I'm not a big fan of putting stickers onto microwaves that say you can't use it to dry your cat. If people want to be stupid, let them.

But hey 12-Factor is a thing, let's roll with it! So in pull request #900 I removed all traces of OSEMs YAML configuration file and now all the settings happen in environment variables.

This was the second gap I had to close to be able to run our app on Heroku. Now all of the sudden things where falling into place and some very interesting things emerged for OSEM.

Continuous Deployment

One of the reasons I have gone down this road was to make it easy for people to try out OSEM. Now once you can run your app on Heroku you can also integrate your github repository and run a continuous deployment (every commit get's deployed right away). That made it possible for us to set up an OSEM demo instance for people to try which always runs the latest code. All we had to do is to make use of data from our test suite to populate the demo (pull request #982) and voila...

OSEM demo by Henne Vogelsang licensed CC BY 4.0

Continuous Review

So a continuous deployment of the latest code with some sample data. Does that sound useful for you as free software developer? I think being a free software developer first and foremost means collaborating with varying people from all over. People I work with on a daily basis and people who I never had contact with before. Collaboration mostly means reviewing each others changes to our code. It's pretty easy as we have rules and tools for that in place. What is not so easy is to do the same for changes to the functionality, the user experience design, of our app.

In the OSEM team we some times attach a series of screenshots or animated gifs to a github pull requests to convey changes to the user interaction and work flows, but this is usually no replacement for trying yourself. Then in the middle of me doing all of this Heroku Review Apps happened. Review apps are instant, disposable deployments of your app that spin up automatically with each pull request.

OSEM heroku pipeline by Henne Vogelsang licensed CC BY 4.0

Now, once someone sends a pull request where I want to review the user experience design I just press the 'Create Review App' button on Heroku and a minute later I get the a temporary instance populated with test data. Magic.

More things to come?

Another thing we might want to replace in the future is how we spin up developer instances. So far we use Vagrant which starts your OSEM checkout in a virtual machine. But nowadays you have to have docker containers in the mix right? Let's see what the future brings.

All in all, I must say, it was a nice trip into the Platform as a Service world. Surprisingly easy to do and even more surprisingly rewarding for the development work flow. What do you think?

10 Tips for Mentoring #FreeSoftware Development

Wed, 16 Mar 2016 00:00:00 +0000

I believe that one of the most important tasks for a Free Software hacker is to bring new people with new perspectives, backgrounds and fresh ideas into the community. That's why I try to make contributing to my own projects as beginner friendly as possible, that's also the reason I frequently mentor people who take their first steps in developing Free Software with programs such as Google's, or Rails Girls Summer of Code.

Just recently I had the opportunity to implement a mentoring program (hey 101) for one of the Free Software communities I'm participating in (hey openSUSE) together with a couple of friends (hey Chris & Cornelius). That adventure required me to think about what I find important about the whole process. I'd like to share those thoughts with you. Here are my 10 Tips for Mentoring Free Software development.

1. Put Your Mentee First

I believe in putting the mentee's needs first. When I mentor someone I'll make an extra effort to place myself in their position. Empathize! Understanding preferences, requirements and circumstances is important. I try hard to comprehend where the mentee comes from and what I need to do to get them to contribute to Free Software. Mentoring is about their, not about my needs, nor the project's I mentor for.

2. Build a Relationship

The most important goal is that you build a relationship based on trust and respect with the mentee. Without this, I believe, everything else you'll do is in vein. Newcomers are overwhelmed and insecure about their skills and it's your job to take that fear from them. Because fear is the path to the dark side! It's the fun times together with you that will matter most to them later on, not so much what code fu-fu you have taught them.

Do not take this lightly. Relationships do not just happen, they need ongoing effort and attention, especially during the early stages. I find that the consistency of the relationship is what's most important. Talking as often as you can, finding a regular schedule where you meet and sticking to it is absolutely necessary, build a routine together.

3. Be a Role Model

Superman by Brian Fuller licensed CC BY-ND 2.0

This might sound like a weird thing from the past and it makes most people uncomfortable but you need to be a role model for your mentee. Free Software development is as much about behavior in a community, knowing what you want, and the ability to convey this to others as it is about pointers, gcc macros and git command line options. You need to ensure that your mentee can follow your example and that you're not passing on your bad habits. You are responsible for developing the competence AND character of the mentee.

4. Lead

Getting into Free Software development can be an overwhelming and daunting endeavor. Precise, reasonable and well-thought-out goals provide a path on which your mentee can take one step after the other into this strange new world. It's your duty that the goals and expectations for the project are actionable, fit your mentee's needs and are technically sound. You lead the way.

5. Audit

I find it important to understand and communicate where the mentee is and how they progress. Setting up regular milestones that I review and we celebrate together provide the opportunity to check and correct the course.

6. Applaud

As a grumpy German this is a tricky one for me. But anyhow, I think it's important to sharpen your senses for your mentee's personal achievements. You recall the excitement the first time you succeeded in setting up the development environment? Remember how invincible you felt once you finally got a hold of git? The time of your first merged pull-request? Now for you and me those things first happened a decade ago and all of this is just the daily grind. For the mentee it's super duper exiting. Make sure you recognize and acknowledge those achievements. Praise.

“In the best, the friendliest and simplest relations flattery or praise is necessary, just as grease is necessary to keep wheels turning. ” ― Leo Tolstoy, War and Peace

7. Collaborate

Whatever challenge you face, remember you are not alone. You mingle on the shoulders of giants together with many other mentors. Make sure you share your experience, your ideas and your problems with them. All of you strive to reach the same goal: getting people into Free Software. Make sure you work together in an open, transparent and friendly manner as part of our worldwide community.

8. Commit

Mentoring requires time, effort and endurance. It will cost you some evening you wanted to go out with your friends, at least a couple of thoughts every day and a lot of nerves because you explain things, that are obvious to you, over and over again. If you are not sure that you are willing to do this then it's probably better that you don't do it.

On the other hand mentoring will give you the satisfaction of being a positive influence in the life of someone else. You will see a new contributor grow into the Free Software community and you will achieve results that matter. You will also get the opportunity to learn something together and sometimes even making a new friend for life.

9. Adapt, be Agile

Bike Tree by Sea Turtle licensed CC BY-NC-ND 2.0

The only thing that is constant is change! You need to adapt your plan, milestones, time line, the technical details, your behavior and attitude during your time. Respond to change, help your mentee to do the same. Be open, courageous, and committed.

10. Don't Follow My Advise, Form Your Own Opinion

All of this is just my opinion and there are many awesome resources about mentoring in general and about mentoring for Free Software development in particular out there. Don't trust me, inform yourself and make up your own opinion. Here are some pointers: