As a software development director, I don’t often get the chance to write code as part of my job. I still read a lot about coding because I love to learn new things, but reading is not the same as doing. A few months ago though, my company went through some significant restructuring and I found myself on working notice. With my old role quickly winding down, I suddenly had some time to do a small coding project that would both benefit my company and allow me to experience some great hands on learning. I got the chance to write my first React application.
The problem
At my now former company, we moved from Bugzilla, Foswiki, and Subversion to Atlassian’s JIRA, Confluence, and Bitbucket server a few years back. As part of the team bringing those tools into the organization, I spent some time figuring out how best to set up permissions so they could be administered and audited from our corporate Active Directory. I was very fortunate to come across some articles and a book by Dan Holme on Role Based Access Control with Active Directory to guide me, and came up with a set of conventions following his recommendations.
Unfortunately, I didn’t complete these recommendations soon enough. By the time I’d managed to meet with all the tool administrators to explain the new plan, there were already many permissions configured that didn’t follow the new conventions. This was particularly the case in Stash (later renamed as BitBucket Server) where the developers had already created dozens of projects, hundreds of repositories, and many hundreds of branches.
The first naive hope was for administrators to follow the “boyscout rule” of always leaving things a bit better than how they found them to gradually move over to the new conventions. The idea was, once we got down to a more manageable number of incorrect permissions, we could make a concerted effort to fix the rest. There were two problems with this idea:
- People are great at pattern recognition – but in our case, with so many incorrectly configured permissions, the pattern wasn’t easy to see and you’d have to refer to a wiki page to remind yourself how to do things correctly. Even the most conscientious admins were making small errors in how they were setting up permissions.
- There was no easy way to tell when the permissions refactoring was complete – or how well it was progressing. Using the web UI to manually check thousands of permissions wasn’t at all realistic.
Automation to the rescue
Atlassian Bitbucket Server has REST APIs that allow you to access permissions information. I wanted to develop a simple auditing tool that would check all the permissions and report which ones weren’t conforming to our established conventions.
There are, of course, many options for implementing a tool like this. You can develop an add-on which gets installed on Bitbucket Server itself and presents information in the administrator screens. This is great for the Bitbucket tool administrators, but doesn’t really act as a good foundation for permissions auditing of other internal applications. Alternatively, you can write a web application in any language you like. For me, I’d been recently learning a little about JavaScript single page web applications and wanted to know more so I decided to use this opportunity to dive in.
My starting point
I started out with a lot to learn. My Javascript experience was limited to following some tutorials in a book on single page web applications and a bit of browsing through code on our old corporate wiki. I’d never used any sort of web front-end framework, had never used Node.js or npm, and didn’t even know what a transpiler was. I’d also never done any functional programming and wanted to see what the buzz was about.
Additionally, I’d recently come across Electron. Electron allows you to write desktop applications in javascript. A number of new and popular tools are written in Electron – including the Slack desktop application and Visual Studio Code (which I was using as an editor). I think that modern web UIs are really beautiful compared to most traditional desktop application framework UIs. By my way of thinking, web applications (and mobile apps) are where most of the new developments in UI and UX are taking place. I loved the idea of using web technologies for desktop apps and wanted to see how it worked in practice.
Key learnings
npm is awesome!
The node package manager, which is packaged as part of Node.js, is great. It’s so easy to use and I love how much it helps in getting your environment set up with all the required dependencies. I also like how dependencies are saved in a simple package.json file so that environments are easy to share. The number of open source packages available through npm is amazing. I got almost everything I needed – other than my editor and Node.js itself – from npm.
Choices, choices, choices…
The huge repository of tools, libraries and frameworks comes at a price. Anyone like me, starting from a blank slate, needs to make a lot of choices early in their project. I spent a lot of time reading articles on X vs Y and trying to identify what were the most modern and popular options to use.
Which front end framework should I use? Angular, Angular 2, or React?Should I use Gulp, Grunt, or Webpack? Should I use Flux or Redux? Should I be using Typescript or Flow instead of ES6 + Babel? Which functional programming library should I use? Underscore, Lodash, or Ramda?
Developers and system architects using Javascript have a lot to keep up with and it seems that the tides are ever-changing. In my case, a lot of the decisions came about because I chose React as my front end framework. A big part of my reason for that choice was that React seemed more naturally compatible with functional programming – which as I mentioned is something I wanted to learn. Once I chose React I tended to choose other components and tools that were commonly used in React applications.
Javascript has some great features
Javascript is often criticized as a poor language so I really wasn’t sure what to expect when I started using it for a non-trivial work (okay – pretty trivial by most standards – but way more than the simple examples that I’d done before). My first big surprise was how much javascript is evolving. ES6 javascript has some great features like arrow functions, the spread operator, destructuring and object literal enhancement. Many of these features are especially helpful in the context of functional programming.
I also really appreciated being able to use a transpiler to make even the most cutting edge javascript features work with older browsers and with tools that might not support ES6 syntax. This seems like a great way to help push the language forward. I wish Python had approached things this way when moving to Python 3.
It was also great to find that even with transpiling and packaging, I could still take advantage of hot reload features to see the effect of code and test changes right away.
Unit testing is time consuming – but worth it
I won’t lie to you. It really does take time to write and maintain unit tests. I’m probably one of the biggest advocates around for unit testing, but that’s easy to say as a manager and another thing to do as a programmer. When I put on my programming hat, I really wanted to do things in a “test first” or “test driven” way – but I found that difficult to do in practice. I found myself writing code and then tests more often than not – though usually in quite small batches at least. It’s an area I could definitely improve in.
Since I was learning a lot while writing this application, I wound up refactoring probably more than most people would. This led to a lot of test refactoring too. I’ve read about refactoring code without impacting tests. Well sometimes that was the case, but other times the tests definitely were affected. For example, removing code duplication also meant removing test duplication – which is a great problem to have – but still work.
The work done refactoring tests was well compensated by the fact that I could easily tell whether or not I’d broken anything when refactoring my code. I was able to do some very significant refactoring with confidence. I was also able to easily isolate and test a lot of special cases that would have been hard to test with the real data – like proper handling of paged APIs and the group name compliance checking code which was somewhat tricky to get right.
A good Lint tool is essential
As flexible as Javascript is, I found it very easy to make mistakes that would result in confusing run time behaviour. Once I installed and started using ESLint it helped me to quickly find and correct a lot of small programming errors – both in the code and in the tests. It also continuously monitored the files I was working on so I would know right away if I broke something.
Functional programming = easy testability
In functional programming, you structure your code so that as much code as possible is side-effect free. The same inputs always produce exactly the same outputs and don’t modify outside state. This makes unit testing extremely easy and also helps to make your code easier to understand. It was odd programming without loops and always taking care not to modify inputs that were passed by reference but to make new copies instead. The resulting code though was clean and worked well with React and Redux.
Asynchronous patterns in React are still evolving
I somehow expected it to be easier – or more elegant – to handle the asynchronous REST API calls in React. React has great examples and documentation on using redux-thunk and I’ve seen other examples that use promises, but it somehow just didn’t seem as clean or elegant as the rest of React so I looked for other options. I wound up using redux-api-middleware which made the code simpler but I’m still not entirely happy with the approach.
For example, the progress spinner I display while getting all of the permissions from the server would tend to flicker while the application was making a large number of API calls. I thought I might be able to solve this with some API call throttling. I tried a couple of approaches using react middleware, but they didn’t have the effect I wanted. I imagine that this kind of thing is a common problem. It would be great to have a package that gives you all the knobs and dials you need out of the box. Maybe it’s already out there. If not, it’s a great opportunity for someone to contribute more complete solutions to the community.
I could almost easily host the app on S3
Since almost everything was written as a single page javascript application, I thought it might be easy to load it into an S3 bucket and host it there. That would save having to distribute the app as a desktop Electron application. It was really easy to repackage the application for S3, and it came up fine when I pointed my browser to the S3 URL. Unfortunately, due to browser security, none of the API calls succeeded when called from an actual browser.
Some other Atlassian products allow you to whitelist applications that can make authenticated calls to the server APIs. This will cause the server API to send back suitable responses to Cross Origin Resource Sharing (CORS) requests. Unfortunately that feature wasn’t implemented in Bitbucket server. Electron disables these standard browser security checks in it’s built-in browser (which is a modified version of chromium) so the API calls worked in the Electron app, but I would have had to do something like use a proxy to get around the issue for an S3 app – and my time was up.
The end result
I’m pretty happy with the resulting code. It’s far from perfect, but as a first application I feel pretty good about it. I think I gained a reasonable grasp of many new (to me) technologies, languages and frameworks and produced a useful tool to boot. I’ve published the code on github for anyone who is curious or looking to do something similar.
On the shoulders of giants
This project wouldn’t have been possible without the many generous people who shared their knowledge in articles, books, and as shared open source packages of useful code. I read way too many articles and books to mention them all, but here are a few of the stand-outs:
- Learning React by Eve Porchello and Alex Banks
Not only did this book do a great job of teaching me about React, it did a great job of teaching me about ES6 Javascript. - Functional Programming in JavaScript by Luis Atencio
This book was invaluable in getting me to think functionally – which was certainly an adjustment compared to procedural and object-oriented programming. - Developing an Electron Edge by Adam Lynch, Troy Mott, Max Gfeller
This book helped me get started with Electron. I was able to quickly and easily get my app up and running. I didn’t dive too deeply into Electron itself, but this book was a great start.
Thanks to all of the above authors and to the many unnamed people who also helped to guide me on this project.
Questions or Comments?
Please feel free to get in touch by commenting on the post or through Twitter or LinkedIn.