2016 was an interesting year for us at DevResults. It was the first year that we laid out a transparent, public roadmap, and so for the first time I get to report back to you on how we measured up against these original targets.
Quick Summary
We added more communication check-ins with clients and have used these to better inform our own decisions. Due to the complexity of the projects we scheduled for early 2016, we released about half of the changes we'd planned. We also released functionality that wasn't part of our initial roadmap and worked on some infrastructure and testing upgrades. Much of what we'd planned for the second half of 2016 will be completed in the first half of 2017.
What We Planned
In my blog post laying out the 2016 roadmap, here is what we planned to work on for 2016:
- Reporting Cycles - Delivered
- Disaggregated Targets - Postponed; date TBD
- Pito & Visualization Overhaul, Dashboard Overhaul - Some Delivered, some In Progress, some Upcoming
- Enterprise 2.0 & a New Data Publishing Model - Delivered
- Data Table Expansion - Postponed to 2017
- Faster Performance & Cleaner Pages - In Progress
We're very proud of how we executed the features we delivered. Reporting Cycles helps clients who have to operate on multiple reporting frequencies much more easily manage data entry and calculation. It's not a particularly flashy or glamorous addition to the software, but it simplifies some rather awkward manual processes a lot of clients were using. Our latest Enterprise offering provides no-hassle linking of related sites, easy and straightforward data publication from one site to another, and a central Indicator Library for each enterprise client. This makes standardizing indicators across countries or offices much easier. And our new Indicator Targets vs. Actuals Graph lays the foundation for enabling more disaggregations and filters in every view of data across the site, besides being really awesome to use.
But as you can see, we didn't hit all of our targets for the 2016 roadmap. There were three main reasons for this:
- We underestimated the complexity and effort involved in Reporting Cycles. We originally anticipated this would take about a month; instead, it took over 3 months and then we had to delay releasing it to avoid impacting the April reporting window for most clients. That miscalculation threw off the rest of the roadmap by about a quarter.
- Visualizations take a lot of work to rewrite. We ended up having to rewrite a great deal of underlying architecture to enable the kinds of visualization our clients seem most interested in, and so the initial visualization we worked on took significantly longer than expected. However, further visualization work should happen on much shorter release cycles because we took the time to build this architecture out thoughtfully.
- Our client engagement processes have changed how we prioritize features. At the start of 2016, we had a lot of clients requesting disaggregated targets. But over the course of 2016, as we refined our methods for seeking prioritization input via quarterly check-ins, we discovered this often was much further down our clients' priority lists than we thought. So we opted to postpone that work in lieu of things that everyone seemed more excited about, such as visualization.
What We Didn't Plan, But Delivered Anyway
This is just as important as noting the targets we didn't hit: we added work to our scope in 2016 that we hadn't originally planned, mostly due to client demand:
- Bulk Imports: create new activities, indicators, etc. in bulk by uploading a spreadsheet
- Bulk Edits: edit existing activities and indicators in bulk
- In-App Help menu: displays help pages relevant to the page you're on, allows you to search the help site, etc.
- Release Notes: see small changes and improvements we've made recently in near-real-time
- Dropbox Document Integration: allows clients with existing Dropbox document storage to link that storage to DevResults Documents
- Merge Duplicate Locations Tool: find potential duplicate locations, merge their data, and remove redundant locations
- Performance and Infrastructure Improvements: these often got tucked into large blog posts, but we continued to rewrite slower-performing sections of the site to get you better performance and reliability
You'll see a common trend here: all of these features help our users do more directly: get questions answered, accomplish bulk tasks that only have to happen occasionally, and remove redundant objects. One of our goals in 2016 was to put more power in the hands of DevResults users and site administrators to complete ever more tasks without needing a DevResults employee's help. It's not that we don't love helping you--we do!--but we would much rather teach you to fish than give you a fish. This has the added benefit of helping reduce the total number of support tickets we get, which makes it easier for us to solve high-priority issues faster.
Larger Lessons Learned
Prioritize, Prioritize, Prioritize.
Before 2016, we didn't have a formalized process for seeking client input on priorities. Clients would request features or functionality and we would diligently document them and make decisions about what to do. But we rarely asked how important that request was in relation to the rest of the requests.
In 2016, we initiated a major process change in our client engagement strategy by reviewing these requests with each client once per quarter, asking them to confirm if things were still an issue, updating them on requests we'd completed or planned to do soon, and having each client rank their requests in order from most to least important.
Without this process, there was always a risk that the squeaky wheel would get the oil--that clients would get a request they made the most noise about, rather than the thing they most needed. By having a simple process to review this on a set schedule, we have been able to make more informed decisions about our own prioritization and have been better able to identify larger industry trends across clients. Increased communication around our roadmap has also helped our clients better communicate to their own users and management teams, providing space for requests from groups who otherwise might have gone unheard.
Plan in Increments that Make Sense.
My initial attempt at a full year's roadmap was basically an experiment to see what planning cycle worked for us. We wanted more transparency and accountability to all of you, but we didn't want to stick too slavishly to that plan at the expense of addressing customer needs. At DevResults, we have always prided ourselves on the fact that we are making tweaks to the software every day. This lets us be very agile and responsive to customer needs. Planning in high detail for a full year--and actually sticking to the letter of that plan--would strip us of this flexibility. A year seemed like a good iteration for planning purposes.
What I've learned here is that planning in detail for many months in advance is not time well-spent. When I tried to go from the high-level roadmap to planning the next three months' worth of work in detail, inevitably some wrench got thrown into the works and I'd have to redo those second and third months' plans. You'll see some of the changes in my approach in the forthcoming 2017 blog post. Here, it's enough to note that we treat product development the same way we treat the software: as a work in progress that is constantly reviewed and refined to try to be the best it can be.
Ship to Learn.
With software development (as in so many projects), there is often this sense that the more time you spend in design, building, and testing, the better the end product has to be. While we've always taken an agile approach to development, in the last year we have deepened our commitment to that philosophy. Rather than trying to identify every use case for a feature and building all of that before we release any of it, we are focusing on delivering smaller, more immediately useable improvements more often.
Now that we have a solid group of Beta Testers and a lot of very excited users (you know who you are!), we see the initial launch of a feature as the starting point for a conversation about how it can grow and develop over time. While it's humbling to admit we don't know everything, it's also been an incredible learning experience working with so many of you during design, testing, and release stages to get your input and ideas. We're excited to build on this momentum in 2017.
Conclusion
2016 was the first full year we've had someone explicitly owning product development and roadmap, which impacted a lot of our processes and responsibilities. It has been a challenging year of learning, but we have grown so much from it. Not only are we delivering awesome features, but we are constantly refining how we get your input and communicate changes.
Most importantly, I am proud to be transparent about what we want to achieve as well as where we misstepped. This is the essence of data-driven decision-making driven by an agile methodology: you set out with a hypothesis about what will work, you deliver products that seem like they'll test that hypothesis, and then you review to refine them. Now more than ever, we are an agile shop. Some of these efforts have proven more successful than others, but across the board, we feel we are on the right track, and we look forward to another year working with you!