Better tools for everyday capture and sharing

With the near ubiquity of GoPro cameras, this feels like a solved problem. But I don't think we are quite there yet. I think that what people most want is a way to record and share their own experiences without having to think about technology at all. This would require not only capture devices and software capable of taking high quality shots, but also automated filtering and rendering algorithms (either pre- or post-capture). The tool should also focus on photos, quick videos, and animated PNGs.

20th century critiques of photography often centered on its unnaturalness. As Sontag and others wrote, it has a way of freezing something in time so that the photographer can gain control of the moment. There may be some truth in this, but we now know that these type of critiques are largely beside the point, which is that the network is at least as important, if not more so, than the photograph itself. This is because the value of a photograph is more about providing others with some context about your situation than it is documenting moments for one's own sake. It is for this reason that longer video is not as useful: it is harder for others to consume (and, again, harder on the network). This also implies that volume can quickly make photographs useless because, again, a recipient can't be expected to sift through a massive archive.

If the main focus is on sharing, then this also implies that quality is somewhat less important. At least, it implies that the bar might be low enough for automated techniques to have a chance.

One key problem is the capture device itself. Drones are problematic for many reasons, not the last of which is that they are likely to be illegal in many outdoor (and indoor) settings. GoPros are close, but are probably too cumbersome for most activities. We need something like a panoramic camera that can be attached flexibly to equipment or clothing that will take photos at an extremely high resolution, and coupled with an automated editing process that will create a personalized and relatively terse archive of photos and animations selected from portions of the panorama at key moments.

Finally, many automated techniques focus on creating a narrative of an event, I doubt this is necessary. I think people just want to feel like they've captured and communicated the gist of an event (in a way that makes them look good, if possible ;)

Smartwatch app for meteor logging

Marking down the path and intensity of meteors during showers can provide useful data for astronomers. It can also simply be a nice way of logging what can be a spectacular event. But I always found it difficult to mark down a meteor's path on a paper sheet. It would be much easier to directly retrace the path through the sky with a gesture. Intensity would be slightly harder to log, but it is usually done with reference to another star or planet of similar magnitude. Perhaps the app could have four stages: 1) gesture path; 2) feedback that path has been accepted with option to manually correct it on an associated phone; 3) point to a reference star; 4) feedback that magnitude has been set.

A similar approach might be useful for other logging tasks as well, for example wilderness tracking, perhaps with speech-to-text to tag the species, etc.

Annotating longform docs on-the-go

I think wearable devices will see widespread adoption only when they can be operated away from phones. This is why I like the idea of wireless earbuds with onboard memory (such as the Bragi Dash). These devices would allow you to load up music or podcasts for a run or hike without having to worry about taking your phone with you (or hassling with wires or brittle Bluetooth connections). I actually never use music while I am on the trails (listening to music can be dangerous as it diminishes situational awareness), but I could see using them for podcasts when hiking through environments that don't hold my interest as well (like cities).

In fact, in that scenario, I would like to add a few features: text-to-speech, document layout hint injections, and annotations. The first two features are derived from SeeReader and are designed to allow one to convert a longform (written) piece into an audio document. The layout hint injections just mean that the system would read out not only the body text of the article, but also note when there is a figure that might be interesting. Obviously you wouldn't be able to look at it at the time, but in combination with an audio annotation feature you could "mark" parts of the document that you want to go back to later. So for example, the text might talk about the growth of fracking in northern CO and reference a map in the document that shows the appearance of drilling sites over time. Saying "mark" could create an annotation to that part of the document so I could check out the map when I'm back from my hike.

Augmented equipment

The 94Fifty ball has been dismissed in some press as a trivial geek toy, but I think it might actually have some utility for young players trying to improve their game. The broader goal of augmenting sports to improve practice and training is something Jono Hey and I explored in our work on ping pong training.

Still, when it comes to adding sensors to sports equipment, I am more interested in improving either the game itself or fans enjoyment of the game. For example, video review during games is something that makes sense rationally but tends to slow things down and interrupt the flow of games. It would be better if the equipment itself was smart enough to detect automatically calls that currently go to review. Instrumented basketballs should be able to detect precisely when they were last touched, and with the addition of player-worn galvanic transponders, we should be able to detect right away who was the last person to touch the ball. This would eliminate all possession reviews as well as some goaltending reviews. The rest of the goaltending reviews could be cleaned up with an arc detector in the ball. Foot-behind-the-line detection (for 3 pointers and less regularly for free throws) should also be straightforward either with cameras or sensors embedded in shoes when there are visual obstructions.

Joey Crawford has his moments. But come on, nobody goes to games to see the refs. More ball less stall.

Better backcountry comm tools

When I am running a backcountry trail or bushwacking off-trail to some big trees (or, in New Mexico, bushwacking while supposedly on a trail), I tend to keep my phone with me, mostly to take advantage of off-line mapping tools. There are many mobile and smartwatch applications that track GPS position, mostly for logging purposes. These are cute, but the functions they provide are relatively unimportant. Consistent networking is (unfortunately, sometimes literally) the killer app for wilderness trekking. There are many satellite systems for SOS beaconing, but the goal is really to send and receive messages before you get into trouble (e.g., if you are canyoneering you'll want to know if there's been rain recently anywhere upstream).

Luckily, there do exist some 2-way satellite systems, such as the DeLorme inReach or the Spot Gen3. I think these kinds of systems could be improved incrementally to be more useful. In particular, this system seems like a perfect fit for push notifications to a smartwatch. It would be nice to be able to configure a service to listen to a particular Twitter stream (e.g., an NWS local) and push important notifications to your device and then onto your watch. It would be even better if the server service would have some basic filtering tools so that you could send only tweets that include "warning" or some other keyword. Even better would be if the NWS has a codified tweet protocol, but as far as I can tell they don't at the moment. This data could also be gleaned via regular web page scrapes from NWS local websites.

Make IoT data fun

The Internet of Things has the potential to overwhelm us with notifications. We are all already bombarded with a variety of usually minor, inconsequential notifications from apps on our phones. Once everyday devices in our homes, offices, cars, and elsewhere begin streaming sensor data to the net this problem could become amplified by orders of magnitude.

Solutions proposed for the coming tidal wave of sensor data often take the form of different types of filters: from simple rules governing what types of notifications should be forwarded on to the user to more complex agents that learn patterns and attempt to detect the importance of notifications. Other solutions rely on displays (e.g., peripheral displays) to tailor the intensity of the notification to the importance of its information.

I propose that outside of absolute catastrophes (e.g., your house is on fire), notifications are simply the wrong model because, in short, people don't like them. There have been many studies of notifications, and most show that users ignore almost all of them (one study showed that they interact with just 6% of push notifications).

A better model for IoT should be apps that people actually like. I would design an IoT app for "down times" that people currently fill with games and social media. When you launch your "Home" app on your phone, instead of seeing a laundry list of notifications to deal with, just show cards. Maybe one card is about water, another power, another food, etc. Swiping away the card dismisses it and sends a small signal to the machine learning system in your app that that topic is less important to you. Tapping on a card opens up more details, or another stack of cards specific to that topic. The app could also learns not only what is or is not important to you, but in some cases also what a good or bad reading is. For competitive people, you could enter local competitions with your neighbors (e.g., who is using the highest % of renewable energy this week). Or, more importantly, show how much money you're saving, or could save, with various changes to your habits. I would show these monetary savings through interactive graphs, a la Brett Victor's explorable explanations.

At least 1 idea/day. Here we go.

I am going to try to get down one idea per day using this blog. "One idea per day" comes from the Seinfeld "don't break the chain" productivity approach.

In the spirit of this effort, the first idea is a blog extension that automatically creates a "chain" visualization. There exist many apps for creating such a calendar generically, but it is always better to have the visualization grow automatically if possible. This could be useful for any event that can be sensed. Apple Watch does something similar with "Activity history", showing a calendar of activity events.

What if you could just attach a sensor to something that you wanted to make sure you did everyday? Let's say I want to learn an instrument. I attach a small sensor to it that is tuned to detect when it is played. I use a mobile app to set my goal every day for a particular sensor (e.g., "play for 20 minutes") then attach it. This would automatically create a "chain" for me on the app.

The problem is, of course, that practice does not make perfect. Perfect practice makes perfect. So how can you tell if you're doing well? Perhaps the sensor has a mic that pics up audio and has some basic analysis abilities to detect quality. If those don't work well enough, clips could be fired off to experts (that is, crowdsource the problem).

Could such a basic sensor be used to track other things in this way? Attach a motion sensor to a bike, to shoes, hats, etc. In this way you could solve the problem with many activity trackers: they fail to track anything that doesn't look like a step. Just attach a sensor button to things (your swim trunks, kayak paddle, etc.) and you'll get at least some data for every activity.