What’s the first question you ask when starting a project? Mine is “What’s our demo?”
I learned this question from going to hackathons almost every month. With the
limited time of a hackathon the question of “what’s our demo” is frequently asked by
your teammates and other attendees. This was most extreme when I participated in Startup
Bus- teams were required to pitch themselves every hour. My team that year couldn’t
decide what our best demo was, or who our best presenter was. But after we tried having
every member present their own way, we agreed that one of us was best at presenting.
And we won the whole competition that year!
This demo-oriented mindset has also helped me and teams I’ve been on deliver large and
complex projects, because you start talking in specifics. In 2012, I wrote about this
in the context of hackathons, but since then I’ve used this mentality
every time I start a new project:
Every unsuccesful hack I’ve made failed because we tried to do too much. No, don’t implement that registration system. No, don’t implement a RESTful API. No, don’t build your app with tools you’ve never used before. Your audience won’t care about those things, so don’t spend time on them.
I always focus on what a teammate once described as “the happy path.” What path would someone take to solve their problem? Build that, and the demo is easy.
If you don’t have a demo planned for your current project, why not? Could you schedule it,
and give yourself a deadline for coming up with something that is presentable?
What’s the demo for your project?
Teaching the Language Server Protocol to CodeMirror
In my ongoing work to improve the Popcode code editing environment, I came
across the Language Server Protocol, which is widely integrated into
desktop coding tools for features like autocompletion. The idea of connecting a language server to
a browser has been tried before, specifically by TypeFox teaching the language server protocol to
Microsoft’s Monaco editor,
but I could not find any efforts to apply that to the CodeMirror editor.
This leads me to today, where I have created a library that connects a CodeMirror editor in a web
browser to a language server, and automatically configures autocompletion, syntax checking, and go to
definition support while being fully customizable. I spent a month creating a hosted demo environment
to explain this functionality- when you connect to that environment, you will get assigned a new container
to run your own language server.
Screenshot of the autocompletion functionality for Javascript/Typescript:
The library code is on Github under the ISC
license. I welcome you to use the library in your own projects and to submit issues
and pull requests you may have! As a solo developer on this, I’m also very excited to see if
this code is useful before investing more in maintaining it.
The frontend of the demo environment is also available on Github here.
This version of the library requires a Web Socket connection to a language server, which is most similar
to the inter-process communication of a language server running on your desktop. If your specific language
server supports these features, they will be available in the browser as well:
Intellisense autocomplete
Signature completion
Hover tooltips
Highlighting matching symbols in document
Linting or syntax errors
Within the same file: Go to Definition, Type Definition, and Find References
There is an example project provided within the library to show how you might configure the library, but
a simple configuration might look like this:
import*asCodeMirrorfrom'codemirror';// You are required to install the show-hint addonimport'codemirror/addon/hint/show-hint.css';import'codemirror/addon/hint/show-hint';// Each adapter can have its own CSSimport'lsp-editor-adapter/lib/codemirror-lsp.css';import{LspWsConnection,CodeMirrorAdapter}from'lsp-editor-adapter';leteditor=CodeMirror(document.querySelector('.editor'),{value:'var code = "Hello world!";',// Optional: You can add a gutter for syntax error markersgutters:['CodeMirror-lsp']});// Take a look at how the example is configured for ideasletconnectionOptions={// Required: Web socket server for the given language serverserverUri:'ws://localhost:8080/html',// The following options are how the language server is configured, and are requiredrootUri:'file:///path/to/a/directory',documentUri:'file:///path/to/a/directory/file.html',documentText:()=>editor.getValue(),languageId:'html',};// The WebSocket is passed in to allow testabilityletlspConnection=newLspWsConnection(editor).connect(newWebSocket('ws://localhost:8080'));// The adapter is what allows the editor to provide UI elementsletadapter=newCodeMirrorAdapter(lspConnection,{},editor);// To clean up the adapter and connection:adapter.remove();lspConnection.close();
This year I want to highlight my favorites from 2018 across books, podcasts, music, and TV.
In 2018 I became more interested in filling in the gaps in my historical knowledge, and
you’ll see a lot of these favorites have some historical elements to them. In 2019, I have
some gaps in my philosophy and classics to fill!
The Wealth of Nations by Adam Smith
I kept hearing references to Adam Smith in economics podcasts I was listening to, so I
decided to read the source directly. While many of his economics ideas are widespread
now, like the use of GDP to measure wealth, I kept finding interesting historical anecdotes
about how these ideas we take for granted got started (like paper currency). The combination
of moral philosophy and economics in one book is hard to find today.
Smil is one of my favorite authors, and this is his masterpiece. He lays out how our need for energy has shaped human history—from the era of donkey-powered mills to today’s quest for renewable energy. It’s not the easiest book to read, but at the end you’ll feel smarter and better informed about how energy innovation alters the course of civilizations.
Evicted: Poverty and Profit in the American City by Matthew Desmond
Another book I found in the 2017 Gates Notes:
If you want a good understanding of how the issues that cause poverty are intertwined, you should read this book about the eviction crisis in Milwaukee. Desmond has written a brilliant portrait of Americans living in poverty. He gave me a better sense of what it is like to be poor in this country than anything else I have read.
The Billion Dollar Spy: A True Story of Cold War Espionage and Betrayal by David E. Hoffman
I had just read an article about spy techniques, and realized I didn’t know much about
how spies worked– so I searched and found this book. It’s based on interviews with the
people directly involved in the most secretive spy operations in Berlin during the Cold
War, and I recently learned that it will be made into a movie.
Black Flags: The Rise of ISIS by Joby Warrick
This is a journalistic and personal story of how ISIS rose to power, and has a detailed
portrait of its founder, Abu Musad Al-Zarqawi. It won a Pulitzer in 2015, and is also being
made into a movie.
The Count of Monte Cristo by Alexandre Dumas
I was looking for an audio book that I could listen to on a series of long flights this year,
and was recommended The Count of Monte Cristo primarily for its length. But the book is a classic for a
reason, and I found myself sucked in to this ridiculous but very entertaining revenge story.
The Indicator by NPR
The funnier version of Planet Money, it’s a short daily economics podcast.
The Great British Baking Show
I’ve been baking desserts for a few years now, and got recommended this show because it’s a
cooking show where the contests are nice to each other.
What Happened? by Gilligan Moss
My favorite musicians of 2018– I’ve been waiting for their album since their remix of Glass Animals’ Gooey,
and they finally released a new EP this year. I saw them perform this album live before its release,
and it was incredible! It’s hard to describe the music, but listen to it here
Radiolab by WNYC
Still one of the most experimental podcasts I listen to, exploring the interesting parts of science,
history, politics, and more. My favorites this year were their podcasts about gender and The War of the Worlds.
This year I ran my first marathon. I would have been running anyway, but having a goal and
deadline made me focus and push myself beyond what I’d ever done before. My work at the
Recurse Center this fall was the same. Now that my 12 weeks at the Recurse Center are over,
I want to reflect on the lessons I’ve learned about myself and my abilities, what went well,
and what didn’t.
Background
When I was laid off from a startup this fall, I wanted to bounce back stronger than before.
I didn’t need to get another job right away, so I had time to focus on training for the
marathon and doing programming projects. At a minimum, I needed to learn the latest technology
to keep working as a software engineer. But I also wanted to complete a project that would tell
a good story about my career and interests, where I would learn a lot.
The Recurse Center sounded like a great opportunity for me:
The Recurse Center is a self-directed, community-driven educational retreat for programmers in
New York City.
We believe people learn best when they take control of their own education and are free to
explore what they’re interested in. RC is heavily influenced by unschooling.
Never graduate. RC doesn’t end after you leave. We have a diverse, active, and engaged alumni
community of over 1,300 smart, enthusiastic, helpful programmers all over the world.
The length of the Recurse Center is up to 12 weeks, and new people start every 6 weeks in a batch.
They also help with finding a programming job, which I thought would be helpful after my batch.
Picking projects
Some of the most interesting work I had done over the last year was volunteering to
teach high school students to build websites. I really liked getting the
students to see the expressive power of code, but also saw how they struggled to understand
all the complexities of it. I had a loose idea that I could improve the tools students are using,
and that if I did good enough work, I could give a talk like two of the best talks I ever attended:
Stop Drawing Dead Fish
by Bret Victor in 2012, and Using Javascript to Teach Javascript
by John Resig in 2015. This led me to brainstorm projects with the restrictions that they:
Were for teaching students to code
Played up my strengths
Used modern technology
Were original
Could be published
Could be completed by myself in three months
I’m happy that I added these restrictions, because the projects I ended up working on were so interconnected
that I was able to plug them into each other in the last week of Recurse. In doing that, I felt like my work
was adding up to something greater than its part, and like my efforts were worth it.
But I also felt like being so focused on my own projects had some downsides, especially in comparison
to what others were doing at Recurse.
Be a part of the community
This may be obvious to some, but it took me a while to figure out how to make the most of the
community at Recurse. The self-organized events didn’t fit my preconceptions of
how to use my time, so I skipped most of them. For example, many Recursers are looking for their
first professional programming job, and hosted interview prep sessions that I didn’t attend.
It was an isolating feeling to be working on my project and not to have anyone who I could
talk through ideas about my work with, and I wanted to fix it.
I started imitating other people who I thought were taking more advantage of the community, by
offering to pair program, debug configuration issues, talk about test setup, and work through
interview problems, which was a lot more fun! Specifically, I was posting and answering more
questions on Zulip, like “Would anyone like to pair on Docker setup?” I found that when I
did this, unexpected people would start conversations with
me, and this finally led to me being able to talk about my project with people and feeling
less isolated.
Share work in progress
I have a tendency to avoid drawing attention to my work, so I thought it would be important for
me to get out of my comfort zone by demoing projects and interesting things I’ve learned at the
weekly Recurse demo nights. Having a mini-deadline every week was something I thought would help
to me live with imperfections in my work. I noticed that most Recursers were also avoiding demos
until they had a working project, which meant that I had no
idea what they were working on, so I would start asking people the same questions I was asking
myself: “What did you learn this week? Can you show me what you have so far? What’s the next
thing that you can demonstrate next week?” Of all the things I did at Recurse, I think this was
the one that was most helpful to others and myself. Sharing my work in progress, and encouraging
others to do so, helped me to know people in the community better, and also helped people get
to know me.
Highlights
Looking back on Recurse, some of my favorite times were the social activities after hours,
and I wanted to remember the fun times outside programming:
D&D nights
People asking me about my marathon
Group lunch at Sahadi’s
Assembling Commander decks for the game collection
Going away Karaoke
What’s next?
Before attending Recurse, I thought that it was a place where everybody would be working on
and talking about their own open source projects, and I applied because that’s what I wanted to
do. I found a much broader range of people– beginners, experts taking a break,
people switching careers, artists, scientists, activists, and more— who all want to be better
programmers. I pushed myself outside of my comfort zone in programming, sharing my work,
collaborating, and being part of a community. I’m sad that Recurse is over, but as they say,
I will never graduate. I intend to be back in the future to share more work in progress!
Text editors and IDEs are no longer the only place to write code. The notebook metaphor has become
a dominant way of writing code for understanding data, online snippet editors are an easy way to try
new web technologies and file bug reports, and online environments can support collaboration with
other people, which can be used equally by software teams or students. These tools mainly fall into a
few categories:
These use the browser as the UI, but are able to be deployed in custom ways
Desktop GUI using a web browser
VSCode, Atom
New tools are using web views to handle text editing
Why do the tools used to teach code look the same as the tools used to analyze data, or to
work on a large project? Why do so few online development tools have integrations with language documentation, test running, or version control? What I found surprised me, which is that each of these editing environments is using one of three libraries, and that no other libraries are yet being used at a large scale:
Supports syntax highlighting, themes, and extensions
Built-in support for language intelligence and IDE features with extensibility
Of these three, the CodeMirror editor is the most widely deployed, though the Monaco editor is quickly growing
in popularity. CodeMirror and Ace are small and fast single-document editors. Monaco is the text editor from
VSCode, which is a larger library with support for more IDE features like language intelligence and refactorings.
Each of these libraries is currently optimized for use on desktop computers, with minimal support for accessibility, although Codemirror 6 will include mobile and accessibility support.
While all of these libraries can support plugins, CodeMirror has the largest selection of existing addons, compared to Ace addons.
It seems like both CodeMirror and Ace are harder to extend than Monaco, as when Mozilla wanted to use the Brackets editor, an IDE built on CodeMirror, it forked the project, which is now used in the Thimble teaching environment. On the other hand, the Monaco editor was able to be directly embedded by the Theia IDE, which was in turn embedded in Eclipse Che. This suggests that the Monaco editor has a well-designed API, but is less suited for diverse use cases like data science tools.
So what kind of APIs would be needed to support diverse kinds of uses, like better integration with documentation, testing, and version control?
Taking inspiration from the good parts of the Monaco API, these features could be split into behaviors and presentation. Tool-makers could control both the behaviors and presentation which their tool would use. This API could be built by a third party, wrapping the libraries in a common API. Having a set of shared presentational and behavioral components for Ace, CodeMirror, and Monaco would be the first step towards a world where tool-makers could tailor their tools to the kinds of users they have.
Conclusion
Web-based coding environments have taken off over the last few years with support from the open source community
and large backers like Microsoft, Red Hat, Adobe, and IBM. These tools are all based off a set of three libraries which don’t have have an API that would support the diversity of users that are working with them, including
professional teams, students, researchers, and non-technical users. In the future, I expect to see coding environments which are as diverse as the kind of code and users that work with them, and I expect that a common API would accelerate that diversity.
I wrote a blog post for Compass about best practices for
using the Google Maps API, including where it’s a good idea
to add functionality using the API, and where it’s a good idea
to keep your code separate.
The code samples are oriented towards teams using angular.js,
but are showing patterns that are easy to translate to other
frameworks.
In the past, I’ve always deployed web apps and side projects to Heroku.
But after being bitten by it one too many times, I decided to get a server
on DigitalOcean and do it myself. It was surprisingly easy.
Setup
First, I created a user called www-data and gave it a home directory of /var/www/.
Inside there, I created this directory structure:
The .virtualenvs folder is used by virtualenvwrapper to manage my environments.
myproject.git is a bare repository that I set up by following
these instructions for git deployment
from Jeff Hoefs. You can customize the post-receive hook, for
example to restart your web process. But first let’s get our process daemonized, so it’ll keep
running if it crashes or the server restarts.
Supervisord
Supervisord is a Python-based process management system.
It’s similar to Upstart, which is
more generic. In fact, I use Upstart to make sure Supervisord starts on boot.
Upstart now ships with Ubuntu, so you just need to install Supervisord. You can install
a pre-built package, or use easy_install like I did. I prefer easy_install, because the
code generated by the package is hard to modify.
$ easy_install supervisor
Here’s an example program specified in /etc/supervisord.conf. By default,
supervisord will start all your programs when it launches.
While waiting for Backspaces to be reviewed by Apple, we decided to build and open source a component popularized by Instagram: live camera filters. If you’re curious, you can read about some of the motivation behind that decision in Dmitri’s post.
Open source projects aren’t usually praised for good design, and our initial versions looked like it. But I knew that we were going to launch this component very publicly, and decided to make some improvements.
As it turns out, I couldn’t find any resources online to make a mockup of the iOS camera. Even the excellent Teehan & Lax resources didn’t have a camera!
In the spirit of open source, I’ve created my own camera PSDs and released them into the public domain.
You should go to a hackathon. You’ll find out quickly if it’s something you enjoy. But if you do, I’d like to help make your experience awesome.
A typical hackathon looks like this: Form a team, come up with an idea, work all night, and finally, present your project. Most last 24 hours. Your goal is to build the best thing you possibly can, starting at the same level as everyone else.
My first hackathon was a year ago. I had no idea what I was doing, but I showed up, met some people, and built something fun. I had a similar experience this past weekend. In between, there were a lot of hackathons, but none as exciting.
What makes these two experience stand out from all the others?
First, the team. The ideal team has a person covering every role. I think four people is best. This past weekend those four people were two developers and two designers. At my first hackathon it was two developers, one person figuring out gameplay, and one person doing sound. The trick is to find a way to work in parallel. You will fail if you don’t.
Second, the idea. Somehow you should find an idea that you’re all excited about. With these two hackathons, we spent a couple hours talking over the possibilities before we found one that we couldn’t stop talking about. Then we spent more time planning out all the features that we needed to build, that would be nice to have, and that we definitely shouldn’t build.
Fake everything except what you need. Every unsuccesful hack I’ve made failed because we tried to do too much. No, don’t implement that registration system. No, don’t implement a RESTful API. No, don’t build your app with tools you’ve never used before. Your audience won’t care about those things, so don’t spend time on them.
I always focus on what a teammate once described as “the happy path.” What path would someone take to solve their problem? Build that, and the demo is easy. Last weekend we used the example of Ken, a potential user we talked to while coming up with the idea. Every part of the app makes Ken’s life easier, so we just had to explain how.
Everyone has a different reason they go to hackathons. Some go for competition, or to build something cool. I go for the people. It’s an opportunity to spend time really getting to know someone, to figure out what motivates them. It’s a community of people that inspires me to be better. I’m still friends with the team from that first hackathon, who have since started a company and moved in together. A hackathon is an opportunity, and it’s up to you to make the most of it.
![Open source iOS camera]](/images/camera-opensource.png)