# Inclusion/Equity and Evaluating Negatives

Privacy, security, inclusion, equity, accessibility, … all of these have an interesting property in common. When evaluation products against such questions you are usually trying to establish a negative.

### To Prove A Negative

Proving a negative requires providing evidence of absence.

If one wanted to *disprove* the claim *“all primes are
odd”* one only needs to prove *“some primes are even”*. The
latter can be accomplished by finding a single counterexample – the
number 2. On the other hand if one wanted to *prove* the claim,
they would need establish that all primes are odd. Another example is
the claim *“there are no vermin in this house,”* proving which
requires an exahustive search of the premises. Once again, disproving
the claim only requires a single piece of evidence – a mouse.

Going back to the beginning, asking someone whether a product is
*secure*, *private*, *inclusive*,
*accessible* etc… fall into the same category. Answering these
questions require proving that there is nothing *insecure*,
*privacy violating*, *non-inclusive*, or
*inaccessible* about this product. These questions are even worse
than the one about vermin from before because it’s not even clear where
to start looking. People who have spent their entire careers studying
the subject would find the criteria difficult or downright impossible to
evaluate.

Despite this apparent futility, there is hope. **Knowing that
the criteria holds under a limited set of circumstances is sometimes
enough**. Often it is the best that we can do.

### Proof By Enumeration

In security and privacy parlance this limited scope is called the *threat
model*.

Absolute security is impossible, so one would start by establishing the specific threats that they would like to deal with. For example when considering the security of a networked application one could start by drawing out a data flow diagram and then going over the different points at which data would be at risk of being compromised. Enumeration of vulnerabilities and threats in this way opens up the possibility of thoroughly examining the attack surface. Thus the previously intractable problem becomes finite.

As our understanding of threats improve, so does the effectiveness of the threat model. Historically, understanding this kind of problem space takes time and many – often costly – mistakes. The boundaries sometimes only become known when someone exploits it.

Similarly, when evaluating a squishy concept like inclusion and
equity one would also start with a well defined scope that lends itself
to examination. Just like before, often we only learn where to look for
problems the hard way. So we come up with methodologies like carefully
curated **dimensions of identity**. Enumerations like that
give us a finite set of things to consider with our finite resources.
Additionally these methodologies also grant us a taxonomy with which we
can organize our hard earned knowledge on the topic.

### Proof by Enumerating Product Outcomes

Enumeration based approaches establish clusters of *possible
product outcomes* which we can then evaluate individually.

As an example, a product that processes pictures of people can be evaluated based on the outcome of handling pictures of people with different skin tokes, gender expressions, body types, ages, hair and accesories, visible disabilities, and lighting conditions. Coming up with these outcomes can be laborious and expensive, but it is tractable. Costs of curating suitably detailed test cases and data sets should amortize across their potentially broad applicability.

### Proof By Construction

But sometimes the set of outcomes do not cluster effectively or are too numerous to exhaustively evaluate. In security this would be similar to asking whether some software system is free from buffer overruns – a question that is theoretically impossible to answer in general if the method of construction is not amenable to analysis. The best we could do is for suitably skilled persons to go through the code with a fine toothed comb looking for potential problems; a task that is both impractical as it is prone to error.

### Proof by Process of Construction

Instead we could try using programming languages that provide certain
safety guarantees: memory-safe
languages. These safety guarantees establish the memory safety of
the resulting behavior as a consequence of the method by which the
underlying software is constructed. Another method we can use is to
continue using an unsafe language^{1}, but invest in
development practices like fuzz testing which give
us sufficient statistical
control over the behavior of our software.

*Inclusion and equity* can follow a similar paradigm where we
establish certain process requirements that also yield a favorable
outcome with a high likelihood. Google’s Building for
Everyone outlines some of these processes criteria such as involving
a diverse group of people during the ideation and design phases of a
product, and involving a diverse group of people during testing. Once
again, these approaches don’t guarantee success, but have a much higher
likelihood of delivering a favorable outcome.

### Amalgam Engineering

Increasingly software systems are built as amalgamations of systems that are themselves complex. The behavior of each constituent is difficult to reason about let alone the behavior of the amalgamation.

To establish some desirable property of the amalgamation, it becomes
necessary to progressively establish the same or minor properties of the
constituents. **The point of establishing these properties is to
yield a simpler more tractable model of a system** that we can
readily reason about. We can then infer or at least approximate the
behavior of the amalgamation based on the simplified models that we
derive of the contituents.

We don’t expect this situation to get better given that many of these contituent parts will be ML models that are too complex for us to understand. As enumerating product outcomes becomes less effective, we will need to rely ever more on process criteria and governance of construction to curb the product outcomes.

The accuracy of the resulting simplified models is all that we will have to grasp in order to have some say how technology will govern us in turn.