Amazon Key

Porch pirates suck. Having anything stolen is a terrible feeling, and having it happen right on your doorstep is even worse. Even though customers know Amazon will cover them in these situations, it doesn’t make up for the feeling of violation customers feel when it happens.

TL;DR How I fought porch pirates, advocated for customer privacy and safety, launched a v1 product, and developed it into a portfolio of solutions.

“How can we securely deliver packages to customer’s homes while they’re away?”

This isn’t a new problem, but Amazon is probably most responsible for increasing porch pirate opportunities. When we started the Amazon Key project, we’d already launched Amazon Locker. While this is a solid offering, there are more than a few drawbacks:

Only available in dense population areas.

Finite capacity, especially during holidays.

You have to remember to pick orders up.

You have to make an extra stop.

Orders get sent back if you forget.

Clunky unlocking experience.

Because of the issues above, you had to find and select a locker for every order. We needed to figure out how to get packages to customer homes. The initial idea for Key was actually a device to secure your porch–a deck box with a smart lock, or a secure bag you could chain to your post. The problem was that not everyone had a porch, ideally with a power outlet, and room to spare for something as bulky as even a small box. And, of course, thieves could just steal the box.

We needed a solution that anyone could use, which meant we needed to narrow down to something everybody has: a door knob. There was some appeal to this plan: a door knob is a sturdy, universal attachment point for a lock’s shackle. We could balance the device so that it hung straight down without tilting too much. We could make the bag fold down to a manageable stored size (using a kevlar-reinforced, cut resistant fabric) when not in use. We ran the numbers, and approximately 70% of packages shipped would fit into a medium-sized bag. What’s not to love? Lots.

Unfortunately, we deemed the concept unviable for a number of reasons:

Customers would have to remember to put out their bag on the morning of the delivery.

The bag interferes with operation of the door.

Once loaded, the bag would inevitably swing into the door as you’re trying to close it.

Children and other household members without access can’t remove the bag.

The device would have to be battery powered, meaning it’s possible the delivery would fail if the device ran out of batteries.

Ok, fine.

Of course, it had occurred to us that there was another option: delivering packages inside. Which means a stranger would be unlocking your front door. While you’re not at home. I’m sure you can imagine why we’d be hesitant.

We were asking for a LOT of the customer’s trust. In return, we made some pretty impactful tenets. right off the bat:

The customer was always in control. They can cancel or deny entry at any time.
The customer was always informed. If something’s going to happen, we’ll let you know.
Everything on video. Amazon Key will not work without a video camera able to operate without its cloud-connected camera. Customers will always has a recording of what happened.

We also had to make an uncomfortable admission: this wasn’t going to be for everyone. By asking customers to trust us this much, we’re significantly narrowing our addressable market. We seriously though about cancelling the project. However, when we asked customers who really needed this, they really, really wanted it, and–to a surprising degree–they really, really, really trusted Amazon.

A difficult setup

One of the most challenging parts of the project was guiding users through the out of the box experience. Whole teams are dedicated to OOBE (out of box experience) for Kindle and Echo products, and that’s basically as easy as connecting to wifi. Our experience needed to support disassembling your front door.

One important part of the experience I won’t get into too deeply here is our partnership with Amazon Home Services. At launch, every Amazon Key kit comes with an option for free expert installation. However, even with this, we knew we had to enable anyone at any skill level to install the product.

To the right is how lock instructions usually look. (Photo credit: Home Depot) That definitely wasn’t going to cut it–our experience needed to be totally guided and customized to exactly what they would be seeing.

We needed to make the best darn lock instructions ever. I’ll walk you through our final wireframe/prototype we used for user testing:

We broke up the instructions into 3 discrete steps. It was important that users had a good idea of the install time, especially since most people don’t have the luxury of leaving their front door lock unasssembled for long spans of time.

Inline help was always available on every screen. Also, Contact Us brought you straight to live chat and a phone number that’s actually answered by a real human without having to dig through a phone tree.

One early finding that spoke loud and clear was that whenever we presented a graphic, it needed to be in the right orientation. When users choose which side their door knob is on in step 3, we orient the rest of the instructions to match what they’ll be seeing.

As we tested, we found that animation was the single biggest installation success factor. Written instructions can only go so far in conveying an idea, especially with inconsistencies in people’s setups. Video was also not as successful. It’s hard to know what to focus on, and there’s too much specificity. When things look like a slight mismatch, illustrations helped users abstract away the differences.

One thing to note: when I created these animations, it was way before I learned CAD or any other sort of 3D modeling techniques. I painstakingly drew these in illustrator on an isometric grid and hand animated/masked everything in Hype. These are undoubtedly the wrong tools, but I’m very proud of what I achieved

Since we were using a 3rd Party lock partner, we tried to change the package contents as little as possible. However, one of the things users got stuck on was measuring the lock bore size. (The two standard sizes are 1/8th inch apart.) We needed to reduce this to a yes/no question so we designed a small pack-in card that let users confidently answer the question.

After finishing the physical installation, we followed typical best practices for digital products. Users deserve a pat on the back at this point–a moment of delight that validates their hard work. We found this in an unexpected place, and it almost didn’t make the cut. There’s basically zero chance that this step fails; in fact, there was heavy debate able whether we should take this step out of our already-long flow. However, the moment where users are able to tap on their digital device and have it manifest in physical action was SUPER satisfying. In fact, when we launched, we observed that users who repeated the step unnecessarily (about ~40%) did so for about 4 full lock/unlock cycles on average!

Testing

We did a few informal spot tests to inform our design decisions as we developed our final prototype. For our final test, we selected a sample of 16 participants, 8 each in Control and Test groups. All participants were internal employees who:

Identified Novice DIYers who have some experience with home projects (painting, installing towel racks, hanging pictures, etc)
Did not work in a technical role
Have never installed a door knob, lock set, or adjusted a door

Participants were asked to remove an existing lock from a door and physically install the Amazon Key smart lock (provided by Yale).

The Control group was given the manufacturer-provided paper instructions.
The Test group was provided a phone with prototype digital instructions, shown above.

We measured for:

Users’ confidence in their ability to successfully complete the task after being shown the instructions, but before beginning. (“I am confident that I can remove the existing lock and install the Prime Access smart lock”, 5 point likert)
Task completion rate and time. (Successful physical installation of the smart lock)
User satisfaction for the installation experience:
- Ease of installation: “The task of removing the existing lock and installing the Prime Access smart lock was easy.” (5 point likert).
- Instruction comprehensibility: “The instructions provided were clear and easy to follow.” (5 point likert).

Results

The test was successful beyond our expectations.

Participants in the test group were twice as confident (3.625 out of 5) in their ability to complete the tasks compared to the control group (1.875 out of 5).
All were able to complete the task in the test group (8 of 8), while just half of the control group (4 of 8) were able to complete the task.
The test group finished the task 60% faster on average (13.9 minutes) vs those that completed the task in the control group (22.3 minutes).
The test group rated the ease of the installation as very easy (3.625 out of 5), while the control group rated it very difficult (2.0 out of 5).
The test group found the instructions very clear and easy to follow (3.875 out of 5), while the control group rated it extremely unclear and extremely difficult to follow (1.25 out of 5).

Notifications and the curious case of TMI

So let’s talk about the elephant in the room. To you and me, the concept of Amazon Key and a stranger opening your front door is kinda… sketchy. But we’re tech people, reading tech blogs, watching tech YouTube, and we’re constantly hearing nervousness about tech’s role in your privacy. Let’s be real, a lot of that is founded.

*One of my favorite hot takes on Amazon Key*

However, I’m also guessing that most of us don’t really need to worry about the problem Amazon Key is trying to solve. Our customers were the kind for whom ordering anything online was a gamble. One person we talked to had to stop ordering online all together because most of the orders she placed were gone by the time she got home from her night shift. For her, and people like her, Key was going to be a godsend.

Have no doubt, we took that trust and responsibility really, really seriously. But we may have overreacted a bit.

You’ll recall our key tenet: The customer was always informed. If something’s going to happen, we’ll let you know. Taking that to heart, we decided that we would let you know when an Amazon Key order was coming.

Below is a normal order lifecycle, starting at the Detail Page (DP), and ending with delivery.

Naturally, when your package is about to be delivered, we’ll let you know, in case you want to change the option.

But we should also probably let you know what’ll happen before you order, too.

While we’re at it, we’ll let you know right after you order too, so you’re not surprised.

Of course, letting you know when the package is on its way is a great time to check in.

And finally, the morning of is important.

And while we’re at it, let’s go ahead and let you know every single day while your package is in transit and every point in the shopping experience.I think you get the idea. This was too much information, and the fact we were concerned over the deliver experience was bleeding over to the customer. It turns out, our customers weren’t worried. After all, they’ve bought this kit, performed an arduous installation, have a camera pointed at their door as reassurance, and most set Key as their default delivery option. They’ve given us every us every indication they trust us to handle it. — And while we’re at it, let’s go ahead and let you know every single day while your package is in transit and every point in the shopping experience.
**I think you get the idea.** This was too much information, and the fact we were concerned over the deliver experience was bleeding over to the customer. It turns out, our customers weren’t worried. After all, they’ve bought this kit, performed an arduous installation, have a camera pointed at their door as reassurance, and most set Key as their default delivery option. **They’ve given us every us every indication they trust us to handle it.**

In the end, we went with just three simple messages. We also refined our language down to make the message more compact, and implicitly suggest the in-door delivery.

Delivering on our promise of security

One part we decided to over message was on the delivery side. We integrated a custom unlock and lock procedure into Rabbit, the delivery driver app. (Our team actually handled all the design and integration work in their app.)

The deliver driver’s device actually connects to the security camera, and the camera communicates with the lock. We don’t let the driver indicate they’ve knocked until the device makes contact. Enforcing the 10 second wait for an answer was a controversial addition–drivers are all about speed–but ultimately was important enough to include.

The circle gestures were to prevent accidental taps. While juggling a package, it’s too easy to accidentally lock the door while delivering.

Finally, we instruct the deliver driver to confirm the door is actually secured.

Amazon Key for Car

Amazon Key In-Home Delivery was never going to be for everyone. We knew that going into this, but after its success, it gave us a platform to build upon. Today, Amazon Key is available for Cars and Garages. We started in on Amazon Key for Car right away. I only stayed on for the foundational stage of this project, but suffice it to say, a lot of people who weren’t interested in in-home were willing to consider in-car.

Setup experience

Shopping experience

Amazon Home and Car Services for Key

The final piece of the Amazon Key puzzle is our integration with Amazon Home Services. I worked on both customer- and provider-facing parts of the Home Services integration.