Tesla: Playtest Engineering III: Testing Cards

This article is WotC-safe.

Hello again, everybody! Once again, a return from a lengthy break, and this time, we're concluding the series about designing an all-commons playtest. Last time, we designed all the commons for our playtest. In the first part, we talked about the overall structure of a playtest set. Now, we're onto the most important part: playtesting the cards, gathering feedback, and using that feedback to improve both the designs and the set as a whole.

Ludevic's Test Subject by Nils Hamm

So, we've designed all of the commons we're going to be using for playtesting. We have the mechanics all represented, the colors all have accounted for some of their most basic themes, and so on and so forth. Now, it's just time to actually try them out!

Going into the playtest, the obvious first step is to have a goal in mind. As discussed in Make Common Cause, every different type of playtest has its own benefits and drawbacks.

• Draft: Great way to evaluate the power level of cards and how well archetypes perform in Limited. Unfortunately it's difficult to assemble 8 players and 24 packs of playtest cards.
• Sealed: The generic Limited format. Not as great of a method of evaluating card power, but it still captures the rough picture of Limited archetypes. Much, much easier to assemble, but still a lot of cards!
• Winchester/Solomon: These also test Sealed, but give you much more control over what cards end up in the playtest, and let you observe the decision-process directly, rather than having to ask about it afterwards - and they are very easy to assemble! Unfortunately, it also has the most divergent gameplay from normal Limited.
• Pseudo-Sealed Constructed: A format that Reuben Covington introduced me to. 40 card decks using no more than 3 of an individual card. This will basically replicate a typical Sealed pool, but it lets players also try out cards they find interesting, and the testers target specific cards for testing.
• Constructed: Actual Constructed decks are basically the same as Pseudo-Sealed but put specific cards through the ringer a little more often.

With the recent playtest, I focused on Pseudo-Sealed Constructed, with the occasional Sealed playtest (using MSE's pack generator), because the goal was to have a sense of how the Limited would play out, but really to focus on individual mechanics. With Pseudo-Sealed, I could ensure I had enough cards of each mechanic in my pool each time that I could truly give them a go.

Epic Experiment by Dan Scott

Once you've determined the goal and format of your playtest, the next step is to actually try it! Find your players, explain to them what you're doing, and give them a brief rundown of the kind of feedback you're looking for. For example, your players should know to keep an eye out for cards they find fun, cards they find oppressive, too weak, too powerful, flavorful, and other topics like that. If your players go into the playtest knowing about this, it'll be far easier for them to remember their answers at the end. In addition, you'll want to ask your players to write down their thoughts during the games.

Do keep in mind that you should find a variety of players. New players, experienced players, and so on. Now, this does depend on the goals of the set - something like Modern Masters doesn't need as much playtesting with new players - but it's always wise to look to a variety anyway. New players teach a designer the importance of simplicity and elegance; experienced players teach a designer the importance of designing a format with depth and creative potential; and both will teach a designer the skills needed to design lenticularly.

Most importantly, you should take notes too! You can sometimes trust your memory, but your memory will totally fail you, no matter how hard you try. Whenever something interesting happens, whenever a card does something unexpected or makes a surprise appearance, whenever a negative interaction or an instance of confusion is witnessed - write it down! Not all of these will be important, but you will regret not having the ability to remember them, and pick which ones were important.

Goblin Test Pilot by Svetlin Velinov

Now, what to take notes about? When you're playtesting, these are the major questions you (and your players, for some of them!) should keep in mind, and record your answers to during and after playtesting:

• Were our mechanics working as intended? Each mechanic had a specific goal in mind, and in some cases - Revolution - different color combinations were intended to use the mechanics differently. This was the most important one of the test, of course, as it was based specifically around testing our keywords.
• Were our cards intuitive or confusing? A card nobody can remember is a problem - an effect everybody misinterprets is a problem. For this, I was specifically eyeing Canisters, as they are quite wordy and have some unintuitive parts.
• Was the format fun? This was less important for our playtest - as it was a very early rough draft, just to see how the mechanics worked in action - but it's still important to notice if it's a specific mechanic's fault that the format isn't fun, or is fun. And if a mechanic, or card with a mechanic, causes games to be unfun - write that down!
• Yes, 'fun' is broad, but that's intentional. Playtesters are often better at saying 'this game wasn't fun' or 'this card was fun', rather than more specific things. As you, the playtest designer, you should see 'fun' in the forms of the following: games that do not snowball - games that have good momentum throughout, rather than board-stalling or accelerating too fast - and interactivity and agency for both players.
• Of course, do remember that with an all-commons playtest, the set will feel funky. Board stalls will be more common, removal will be rarer and harder to use, and so on. That's just a part of the all-commons experience, and learning to filter out the 'natural' all-common feel from an unnatural problematic feel is hard, not going to lie.
• What cards were way too bad, and what cards were way too good? It's okay for some cards to be pushed, and some not to be - and remember, your designs should be the pushed ones in a playtest - but if a card cannot be reconciled with the rest of the set no matter how much you change it, then it's a problem. In Reuben Convington's words - "Is it developable?"

For example, here are some notes I took from my playtest, both during and after:

"Lots of 2-tap ability go well with Canister. Tap/untap subtheme? Lorwyn/Shadowmoor/Theros inspiration? Redeem a theme?"

"1-drop Justice feels strong but fair. Tough to deal with but never feels oppressive."

"Prowess Rev. is weak but interesting w/ thopter. CA hard to find in UR. Need bigger consistent burn."

"Fugitive Freelancer plays very well. UG goes fast but not consistently. Revolution ramp never feels right."

"GR Rev doesn't work. Curve is off in G/R and R/W rev. G/W rev near impossible. G/R didn't end up as midrange as hoped. Challenge Allegiance plays well."

"Canister confuses often due to poor templating on my part."

"Crew plays very well but poses board complexity. Lose mechanic Reuben suggests cycle vert."

Simple notes, as you can see. I mention cards I like, cards that perform well, themes I notice, mechanics that impress, cards that I want to keep an eye on, and so on. Some of the themes relate to fun - I often lacked answers in UR, and couldn't keep up momentum consistently in GR - and others relate to intuition - the Canister confusion I had in an independent playtest of my own with someone who hadn't seen the project. For the most part, though, it went well!

Parallectric Feedback by Mitch Cotie

So, we've gotten our feedback, taken notes on our observations... which means the next step is to use these to improve the cards! My first step is always to take any suggestions from players into account, and decide whether to pursue them or not. Reuben was right about Crew - we demoted it from keyword, since it plays very well, but really doesn't need more than a vertical cycle of cards in the set. My friend suggested a change to Canister wording that I didn't like much, but I'm still looking for a better way to do untapping Canisters without confusion.

Then, I look at specific cards I mentioned in my notes. The 1-drop Justice got altered so that its interesting play is maintained, but it doesn't get as big as fast. Fugitive Freelancer gets a stamp of approval, as well as other cards. The Prowess Revolutionary is being watched, to see if it has a deck in UR Thopters, or whether we can change it to be more playable. 

The next step is to look at mechanics and broad 'collections' of cards. For example, I noted that Revolution's curve needs work. I've noted this in the file and, since I don't need to design more cards but rather change the costs of cards, I intend to address this in the next playtest file I make.

Lastly, after all other changes have been applied, double-check the entire set. Some of these changes could begin to cause overlap, redundancy, or other issues with the rest of the cards in the set. You might end up with too many cards at 2CMC in blue, or a change to a card that seemed intuitive could cause problems when it interacts with another card in the set. Make sure to address these problems, and change the other cards accordingly, to fit the changes you've applied.

Now, here are the two guidelines one should follow when updating and changing designs after a playtest, and the big take-away of this article:

• Be ready to accept and try anything. Mechanics dear to your heart may crash and fail. Bold experiments may baffle and confuse. And cards you were ready to cut can become beloved all-stars.You don't have to follow through with every suggestion or try to fix all feedback, but you should definitely give each one serious consideration!
• Always be changing. Every playtest should be different than the last, and when you look to individual cards or mechanics, your first thought should be "How do I improve this?", not "Is this okay as it is?" This doesn't mean change a lot. It means that your playtests should always be done with the intent to improve, and that you can't afford to get too attached to things and settle.

Keep those in account while reviewing your notes and the cards, alongside the other advice here, and you'll be well on the way to some major improvements in your file with each playtest!

Ludevic's Abomination by Nils Hamm

And that concludes this design series on designing an all-commons playtest, from start to finish. Thanks for reading along, and I hope it was both informative and fun. Until next time, everyone!