RULES-IS Immune-network inspired machine learning algorithm
D.T. Pham and A.J. Soroka
Cardiff University, Cardiff, UK
Immune System
- Biological system for the identification and elmination of foreign bodies
- To be able to do the IS must have:
- Ability to learn
- Ability to recognise antigens
- Ability to remember
RULES-IS
- Rule extraction system based on Immune Network theory
- Widely used Immune System theory in AI
- Learning and classification not tied together
- Platform independent
- Generate rule sets for use in any Expert System Shell
Representation of Attributes and Rules
An attribute => antigen (the data that is used for learning)
A rule => antibody (what is used to classify an attribute/antigen)
Antibody:
| Leak |
#1.1>9.1< |
96.5 |
shade |
* |
Content:
| class |
range 1.1≤#≤9.1 |
number |
string |
wild card |
Immune Network
Used to store the antibodies and antigens used to produce rule set.
Antibody (Rule) generation
Two methods exist to produce an antibody:
|
Ab:
|
Val.A
|
#1.2>2.4<
|
3.1
|
X
|
*
|
1
|
|
Ag:
|
Val.B
|
2.5
|
4.1
|
X
|
Z
|
1
|
|
New Ab:
|
*
|
#1.2>2.5<
|
#3.1>4.1<
|
X
|
*
|
1
|
- Direct from Antigen (where no Abs exist): The Antigen or training example becomes a rule
Decomposition
Important feature of RULES-IS
- Facilitates generation of Antibodies (Rules)
- Helps prevent creation of rules that misclassify
- Allows wildcards to be used
Uses Immune Network
- When an Ab is found that misclassifies
- Drills down to antibody that doesn't misclassify
- Antigens released go back to list of examples for training
Overall algorithm
In summary the algorithm
- Memory of any previous immune network is loaded
- An antigen is then selected from the list of antigens
- Antigen matched against any antibodies
- Best match selected used to form new Antibody
- Check that the Ab and Ag classes match
- Decompose if there is a misclassification
- Form new rule if everything is ok
- Repeat until no training examples exist
Results - Season classification
| Algorithm |
No. Rules |
Accuracy % |
| RULES-IS |
5 |
100 |
| C5.0 |
4+1 |
100 |
| ID3 |
5 |
100 |
| ILA |
5 |
100 |
| RULES-3 |
5 |
100 |
| RULES-4 |
5 |
100 |
Results - Iris Data Set
| Algorithm |
No. Rules |
Accuracy - Training |
Accuracy - Unseen |
| RULES-IS |
4 |
97.14% |
95% |
| C5.0 |
5+1 |
100% |
91.25% |
| RULES-3 |
14 |
100% |
97.37% |
| RULES-4 |
10 |
97.14% |
93.2% |
Conclusions
- Proved comparable to existing rule generation algorithms such as C5.0, ID3 and the RULES
- Particularly wrt the size and accuracy of the rule set created.
- Has shown Immune-based methods suitable for rule extraction
