Data Mining, Criminal Sentencing, and Analogy

Florida will be using data mining to suggest the program to which each juvelile delinquent is to be assigned.--which extracts rules from data, just as the machine learning techniques such as ID3 do-- Data mining is just a sophisticated version of these techniques.

This is not new. People have been talking about using techniques such as regression to predict recidivism and help judges make good sentencing decisions. (I won't take the time right now to do the research.) Many states used very specific guidelines--I had a programming course with many students from my University Law Enforcement and Justice Administration Departments. I had the students program the guidelines as a real-world example of a complicated if statement.)

The question is which is most likely to sentence a juvenile correctly, the lowest recidivism rate for the lowest cost in incarceration, the predictions made by a set of rules, a prediction made by an expert such as a judge or criminologist,, or a prediction from a randomly selected jury. A relatively simple study could do this--have all three make a prediction, sentence the juvenile to one randomly and watch the outcomes. WorThat is a question that could be ank going back to the sixties show that statistical approaches predict recidivism better than "clinical experts." (Caroll, John S., "Judgments of Recidivism risk: Conflicts between Clinical Strategies and Base-Rate Information", Law and Human Behavior volume One, Number Two 1977, page 191-198

The challenge of participatory democracy, just as it is for conventional sentencing, is how to incorporate the decision making. The Participatory Democracy could develop a set of rules for sentencing while being given the results of these regression tests. These then would be applied with little or no discretion to each individual offender. Or each offendor could be sentenced by a participatory democracy jury, with the juries receiving a prediction or data from the statistical model or data mining model. However, sadly, Kahnmean and Tversky showed that individuals did not properly use statistical data. The alternatives for jurors to use for sentencing are similar to that for tax decisions--and if prison and jail availability is a limited resource, society could independently decides how many slots are for rapists, how many slots for armed robbers, etc. and the participatory democracy jury could vote for which armed robbers would be assigned to those slots.

There are many references in Dr. Carroll's article to literature on human judgment in using statistical models in general and its use in criminal sentencing decisions in specific. I will track many of these down for a future Thoughtful Thursday.

Thoughtful Thursday: The ID3 Machine Learing Algorithm

ID3 is an algorithm to generate a decision tree from a set of examples. It is used in machine learning.

Many legal codes are some sort of function of several parameters and are of a nature that can be defined as a decision tree. For example, one is making the gun control law. Some of the parameters might be the type of gun. (Is this a BB-gun, an antique gun, a non-functional gun, an assault gun, a pistol, etc?) Is the weopon concealed? And there are the characteristics of the owner of the possessor. Are they mentally ill, convicted of a crime, have an order of protection against them, etc. Was the person's life threatened? Does the person job need it, e. g. security guard. And lastly, what is the place: a school, bar, a place where beer is sold such as some supermarkets.

The answer could be a simple boolean, "permitted" or "not permitted" Or we could get a classification such that the possession might be a misdemeanor or a felony. Many states have categories for felonies. For example, in the State of New York, a class A felonly, the term is life imprisonment, for class B, the term for a class B felony does not exceed twenty-five years, etc. (There are other special rules for dispositions, such as those who have prior convictions or are youthful offendors.) Then the bulk of the penal code defines offenses to be a class D felony, a class B felony, etc.

A possible decision tree

Gun Type	Location	Age	Personal Status	Penalty, if any
antique				No Offense
BB Gun				N Offense
Pistol		15-20		A Class Misdemeanour
Pistol		20-30		B Class Felony
Pistol		30-50		Felony
Pistol		50-70		A Class Misdemeanour
Rifle	Bar			Class B Felony
Rifle	Schol			Class C Felony
Rifle	Place where Beer is Sold			Class A Misdemeanour
Rifle	other			No Offensse
Assualt				Class B felony
Rifle	Other		mentally ill	Class D Felonly
Rifle	Other		Convicted of a Felony	Class C Felony
Rifle	Other		Other	Class B Misdemeanor

An ID3 algorithm would take a series of examples such as the one in the table above. It would create a decision tree that would give the same answers as the examples. It needs to pick which parameter to use for the root node--in the above tree, it was Gun Type. It picks one that divides a lot of examples between thee categories. Often the formula from information theory is used. In the above example, the tree was precisely determined from the input data. That is not always true. In a diagnosis application, there would likely be lines that contradict each other. Or there would might be several lines with very similar attributes. All might have the same classification or penalty, except one. Is that due to a real difference, or just a random error?

A	B	C	D	Result
1	2	3	44	ii
1	2	4	44	ii
1	2	5	44	ii
1	2	3	45	ii
1	2	4	45	ii
1	2	5	45	ii
1	2	3	46	ii
1	2	4	46	ii
1	2	5	46	iii

There is a danger of overfitting when there are lots of examples. Thus, a pruning process removes those divisions that don't have much information gain.

Mr. Yeluri set up an example to illustrate these ideas. Since it is long, I put it at the end in an appendix.

In participatory democracy, we do it a little bit differently. For each node in the tree, the users vote on what the next parameter should be. Thus, at the start the individuals would vote on which of the above questions regarding place, type of gun and person should be asked first. Then, the group for each resulting category would be classified again. The individuals can vote to decide for any node when it comes up to classify it once and for all. E. G., if the category of assault weopon shows up, individuals might vote to always ban it and for the category of BB-gun, they may vote not to consider any other factors and always permit it. This is a leaf node and the people will then vote on the classification from the possibilities available.

We established the following terminology, for the input to our experiment:

title

a thing on which one chooses, like gun type

attribute

one of the possibilities for a title thus gun type could be antique, bb, assualt, rifle, assault or shot gun

node

this corresponds to series of pairs of parameter and type.

for example, the voting might bring us to making a decision on what to do with those caught with an antique guns, a history of mental illness, and who are sixteen to twenty years old. This is a list of title (nature of gun) and type, antique, mental illness status, and type of history of mental ilness and age, which is divided into types or ranges. this is defined as

of root node

age top of the tree which coresponds to no titles and parameters selected so far.

The voters first vote on which title to choose as the root parameter. Then, they vote for each attribute of that title, how next to classify it. as ech of thse achieve quorum, each of these nodes becomes accessible, that is, the voters might vote on what title to use for gun-type= antique, choosing from the remaining titles of personal status, age range, location. then, they vote on what title to use for gun-type="bb" gun-type="pistol" gun-type="rifle" gun-hattype="assault." Each creates a new node which is marked accessible; and individuals vote for each in turn. One of Masters students, Mr. Chaitanya Yeluri, just implemented this. More details are in his masters project.

After the root node, the users can assign a penalty. Observe in the figure that we immidetely assign "no offense" to those possessing antique guns. on the other hand, there arthree levels until we get to gun type ="rifle",m location="other" and "mental health status" = "felony conviction." at each node, the users have the option to "vote now" and if that gets more votes than any title, the users vote on a penalty for same.

How would we apply this to tax law? The ID3 algorithm is not designed to deal with cases that produce a continuous function, like the amount of tax that the taxpayer should payer. (Note, ID3 is used when there are input parameters that are numeric. The algorithm sorts the examples in numeric order of the parameter. It then finds the decision pont that gives the greatest information theory gain.)

Thus, for a tax system by ID3, at each leaf node, there would per be a formula. The titles in a tax problem might be various income ranghe atues, the number of children, marital status. And then the users might vote for those unmarried head-of-household with three children and in the range $20,000 to $40,000 to make the tax = $500.00 + 0.10*(Income - $20,000). That is $500.00 on the first $20,000 plus ten percent of the remaining income. They might put other components of the formulae in for the amount of state tax they paid, the mortgage interest, etc.

Reference

, Quinlan Ross, Programs for Machine Learning, Morgan Kaufman, San Mateo CA 1993

Appendix, The Information Theory Formula

We have three titles, A, B and C. The possible choices, or parameters for each of these are:

A	F, G, H
B	I, J, K
C	M, N, P

Assume the input examples are:

A	B	C	Classification Yes /No
F	I	M	N
F	I	N	Y
F	I	P	N
G	I	M	N
G	I	P	Y
G	I	M	N
G	I	P	Y
G	I	N	Y
G	J	M	Y
G	J	P	Y
F	J	M	N
F	J	N	Y
F	J	P	Y
H	I	N	N
H	I	N	N
H	J	N	N
H	I	P	Y
H	J	M	Y
H	J	N	N
H	J	P	Y
H	K	M	N
H	K	N	Y
H	K	P	Y
F	K	M	N
F	K	N	Y
F	K	P	Y

The information content of the entire sest is 0.3149. We try classifying it on the basis of A, and we get an information content 1.3328. And it happens by coincidence that if we tried B for the root node, we also get 1.3228. (The three nodes for a would have been:

A	B	C	Classification Yes /No
F	I	M	N
F	I	N	Y
F	I	P	N
F	J	M	N
F	J	N	Y
F	J	P	Y
F	K	M	N
F	K	N	Y
F	K	P	Y

giving an information content of 4/9 log ₂ 4/9 + 5/9 log ₂ 5/9.

A	B	C	Classification Yes /No
G	I	M	N
G	I	P	Y
G	I	N	Y
G	J	M	Y
G	I	M	N
G	I	P	Y
G	J	P	Y

giving an information content of 5/7 log ₂ 5/7 + 2/7 log ₂ 2/7.

A	B	C	Classification Yes /No
H	I	M	N
H	I	N	N
H	J	N	N
H	I	P	Y
H	J	M	Y
H	J	N	N
H	J	P	Y
H	K	M	N
H	K	N	Y
H	K	P	Y

giving on an information content of: 5/10 log ₂ (5/10) + 5/10 log ₂ (5/10) .

By comparison, the sample sets for classifying by C are:

C="M"

A	B	C	Classification Yes /No
F	I	M	N
G	I	M	N
G	J	M	Y
F	J	M	N
H	I	M	N
H	I	N	N
H	J	M	Y
H	K	M	N
F	K	M	N
G	I	M	N

And, if we tried to split this by A, we would have:

A	B	C	Classification Yes /No
F	I	M	N
F	J	M	N
F	K	M	N

A	B	C	Classification Yes /No
G	I	M	N
G	J	M	Y
G	I	M	N

A	B	C	Classification Yes /No
H	J	M	Y
H	K	M	N

giving a total information content of 0.6438. If we split C="N" by B

A	B	C	Classification Yes /No
F	I	N	Y
G	I	N	Y
F	J	N	Y
H	I	N	N
H	J	N	N
H	K	N	Y
F	K	N	Y

Then, there are two possibilities breaking it down by A

A	B	C	Classification Yes /No
F	I	N	Y
F	J	N	Y
F	K	N	Y

A	B	C	Classification Yes /No
G	I	N	Y

A	B	C	Classification Yes /No
H	I	N	N
H	J	N	N
H	K	N	Y

This gives us an information value of 1.5579.

Or we can split it by B, giving us:

A	B	C	Classification Yes /No
F	I	N	Y
G	I	N	Y
G	I	M	N
H	I	N	N

A	B	C	Classification Yes /No
F	J	N	Y
G	J	P	Y
H	J	N	N

A	B	C	Classification Yes /No
H	K	N	Y
F	K	N	Y

This gives us an information value of 1.6253.

The resulting tree chosen would be given by the below:

Options for Participatory Democracy and Sortition for Taxes

Possible Mechanisms for Participatory Democracy for Taxes

I proposed and our group at Western Illinois University are implementing techniques for peole to vote on the laws that constitute the tax code. The tax code could be represented as a decision tree which in the end leads to the amount of tax or a simple formula such as a linear function of income that gives one tax. A decision tree is a set of divisions, each of which will have other divisions. For example, we might vote to divide people on the basis of the number of children. Thus, we would look separately at those having zero children, one child, two children, three or four children, five to eight children, etc.

Then, we might look at the ratio of earned (wages) to unearned income (bank interest, dividends, bond coupons). Thus, the tree might have a division for those having no children and eighty to one hundred percent income. Another branch for those having no children and sixty to eighty percent income, etc.

Then for each of these divisions, we would have a formula or graph relating income to division.

At each stage in the process, individuals would vote on what divisions to make and eventually the ratio of income.

Another student is working on applying genetic algorithms to determing a tax code.

But do we need rules? Do we need a tax code?

We simply could say that each individual and each entity would go before a jury which would determine the tax they pay. How can we make it less arbitrary?

1. Tax rates would not vary by more than ten percent year to year without a supermajority. To transition to that system, we would start with whatever tax the entity paid under our complicated tax code. Thus, if a firm paid four millions in taxes the previous year, their tax this year would be between 3.6 million and 4.4 million. However, sixty percent of the jury could vote to change it by twenty percent, in the example from 3.2 million to 4.8 million. Seventy percent could vote to change it by fourty percent, etc.
2. Several entities could be grouped together for comparison. One possibility is to group them randomly. Thus, a jury would see a disparate group of entities, say
   1. a middle-class individual
   2. a working-class individual
   3. a financial institution
   4. a factory
   The jury would vote on a tax rate for each entity. It doesn't follow that the corporations would pay a higher rate than the individuals. The jury might find that the corporation truly is a public-spirited organization committed to sustainability. And it might find that the individual is a fuel hog in taking unnecessary trips, and not taking care of self. (See my blog entry on the badness tax.)
3. The above scenario assumes no attempt to group elements. One certainly could group members. This can done by rules. That is, we could vote as discussed at the beginning for classifications. We could vote on dividing by corporation, partnership or individual. We could vote to divide by their net income, gross income or number of employees.. Thus, one would not be comparing small businesses with large businesses. But we wouldn't vote on a tax rate for each category, simply a total amount of revenue to be collected from each group.

   A sample of let's say twenty individuals or entities or juries would go before the tax jury. The tax jury would know that they have to collect a specific amount revenue from each individual. (The computer would divide the total to be collected in each category by the amount of revenue).

   Example, we vote that we want to group all those married individuals earning between sixty-thousand dollars and eighty-thousand dollars and having two children in one group. We would look at the total income earned and decide that gather all such people should pay twenty billion in revenue. Assume this group was two-million families. Thus, on average each family would pay ten-thousand in taxes. And thus, each group of twenty would pay $200,000 in taxes. The jury would tnen adjust the $10,000 that each should pay based upon all kinds of other factors: how much have they given in charity, which have high medical bills or suffered other disasters this year, etc. etc.

   Each family would be given a chance to explain their financial situation and any reason why they should be given special consideration.

4. The alternative to rules for categorization is clustering. This would introduce a second type of jury. This jury would be given pairs of individuals. They would get financial information for the two individuals in the pair. These jurors would indicate how similar they are; not how much taxes they should pay.

   There are many algorithms available that cluster items into similar groups icnluding Self-organizing maps. In two or three spatial dimensions, this would be groups of points that are very close, forming a clump on a scattergraph. This could be extended to a tax situation in that the software would treat numbers such as number of children, incomes, as spatial dimension and place each taxpayer as a point in the "n-dimensional space." The clustering algorithm would find groups of tax payers that are similar in input characteristics.. The taxpayers would go before groups of the first juror types to explain those special tax considerations that would be lower than individuals that are similar.

5. We have many algorithms to take sets of example data and create a function out of them. Thus, the jurors could rate several tax payers as to how much tax they should pay. This, of course, assumes that the characteristics that determine how much tax an individual or entity should pay are all quantitative or captured by the collected parameters (income, medical expenses, etc.) Are we better off allowing people to present these issues and construct the rules interactively and collaboratively, or merely say what the tax should be for various tax payers and construct the rules mechanically.

Computer scientists and statisticiians have developed many different ways that one could learn functions from example data. These relate from such standby's as multiple regression to the new methods such as neural networks, and machine learning techniques. Thus, assume that we have juror ratings of one thousand tax payers. We can apply these techniques to generate a function between the parameters and the tax assigned. Obviously, one could replace the jurors by the formula or alternatively, people could use such capabilities to estimate how much tax they would have to pay and when.

(This is similar to the various services that report jury verdicts in tort litigation to help trial lawywers decide when and for how much to settle their cases.)

In simulations, one can compare the results to a Lindahl Equilibrium, which I will present later in a Thoughtful Thursday and discussed earlier in the discussion of Genetic Algorithms.