Full Stack Antelope Classification App

Author

Rubanza Silva

Published

March 20, 2023

Antelope Classification Application - Introduction

Uganda, East Africa is blessed with a rich ecosystem that accommodates various kinds of species of Animals.

On a recent drive through the savanah adjacent Queen Elizabeth national park ( right before Katunguru), I happened to spot a number of antelopes inspiring me to create this antelope classifier

In this example,i build an antelope image classification model which provides a prediction of the label showing which class the antelope belongs too and its corresponding confidence.

I then deploy the antelope classifier model to hugging face spaces and build a web interface using gradio. The antelope dataset used to train the model is scrapped from duck duck go.

After you train a model, Various people and departments from software enginners, business people etc will need to use the results from your model in order to drive business value, inform decision making, build into their current processes etc.

If you want people to interact with your model outside of a juypter notebook will need to deploy your model to a web application building a UI on top of the model that can anyone can interact with.

To do the above, we host our model on Hugging face Spaces first and then use gradio to build a web interface for the model but first we start with prequisites i.e. data collection and building the model.

This notebook for this can be found on this github link

Imports

Import fastbook which sets up all the dependencies youll need to complete this notebook. If you run the notebook in google colab.This code cell mounts google drive in colab

Code

#hide
! [ -e /content ] && pip install -Uqq fastbook
import fastbook
fastbook.setup_book()

!pip install gradio==3.0
# !pip install bentoml
!pip install nbdev

#hide
from fastbook import *
from fastai.vision.widgets import *

::: {.cell 0=‘e’ 1=‘x’ 2=‘p’ 3=‘o’ 4=‘r’ 5=‘t’}

Code

import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
from fastai.vision.all import *
import gradio as gr

:::

Note that i use #| which is an nbdev trick that later enables us export the code in those cells into a general .py file.

We declare a default export at the top with the “.py” file being named app.py

Data collection

For any machine learning project, one of the most important and starting points is the data. For the above classification task, We search and scrape the web using duck duck go for images of antelopes. Duck duck go currently doesnt require any API keys to pull images from the web.

In the cell below, we search duck duck go for one image, download it then display just to test our scraping functionality

Code

urls = search_images_ddg('Uganda Kob', max_images=1)
urls[0]

'https://cdn.outdoorhub.com/wp-content/uploads/sites/2/2017/03/uganda-6-800x533.jpg'

Code

dest = 'images/ugandaKob.jpg'
download_url(urls[0], dest, show_progress=False)

im = Image.open(dest)
im.to_thumb(128,128)

Above is a preview of the image we just downloaded resized to 128 x 128 px. All images are converted into a tensor then fed to the fastai dataloaders. Images passed to a dataloader must be tensors, hence they are all converted to tensor and must be the same size. To ensure this, We always apply the same transformations across all images.

Define search terms

Define search terms that our chosen search engine will search for. A path where the downloads will be stored is defined. In this case, we shall be searching for images with the defined search terms for types of antelopes below

Code

#search terms and path
#search times should be on one line
antelope_types = 'Eland', 'Greater Kudu', 'Hartebeest', 'Oryx', 'Defassa Waterbuck', 'Sitatunga', 'Impala ', 'The lesser Kudu', 'Grant’s Gazelle', 'Reedbuck', 'Uganda Kob','Forest duiker','Harvery’s red duiker', 'Blue duiker', 'Peter’s duiker','Black fronted duiker','Grey duiker','Oribi','Klipspringer','Guenther’s dik dik', 'Bates’s pygmy','sable', 'roan', 'nyala', 'bushbuck', 'tsessebe', 'steenbok'
path = Path('antelope')

Code

from time import sleep

for o in antelope_types:
    dest = (path/o)
    dest.mkdir(exist_ok=True, parents=True)
    download_images(dest, urls=search_images_ddg(f'{o} antelope'), show_progress = True)
    sleep(10)  # Pause between searches to avoid over-loading server
    download_images(dest, urls=search_images_ddg(f'{o} antelopes in a herd'), show_progress = True)
    sleep(10)
    # download_images(dest, urls=search_images_ddg(f'{o} antelope low resolution images'))
    # sleep(10)
    resize_images(path/o, max_size=400, dest=path/o)

/usr/local/lib/python3.9/dist-packages/PIL/Image.py:975: UserWarning: Palette images with Transparency expressed in bytes should be converted to RGBA images
  warnings.warn(
/usr/local/lib/python3.9/dist-packages/PIL/Image.py:1038: UserWarning: Couldn't allocate palette entry for transparency
  warnings.warn("Couldn't allocate palette entry for transparency")

Code

#Navigate to image directory to check out downloaded images
!ls antelope/Oryx

Code

im = Image.open('antelope/Oryx/d7a8f21c-9373-420d-96af-9e3591aa93c9.jpg')
im.to_thumb(128,128)

Figure 2: Preview of image from our downloaded dataset

Remove failed images

We use a verify_images function to iterate through our downloaded dataset, check for corrupt images ( by finding the images that cant open ) and removing the failed links

Code

failed = verify_images(get_image_files(path))
failed.map(Path.unlink)
len(failed)

Define datablock and build model

###DataBlock We need to put the data in a format that can be used to train our models by creating a dataloader object. A datablock API helps us customize our dataloaders object by telling fastai the four things we need to define a dataloaders class.

The type of data we are working with ( blocks )
How to get the list of items ( get_items )
How to label these items ( get_y )
How to create a validation set ( splitter )

Code

antelopes = DataBlock(
    blocks=(ImageBlock, CategoryBlock), 
    get_items=get_image_files, 
    splitter=RandomSplitter(valid_pct=0.2, seed=42),
    get_y=parent_label,
    item_tfms=Resize(128))

Code

dls = antelopes.dataloaders(path)

Dataloaders class provides images for processing in batches of a few items at a go. Below is an example of such a batch i.e batch of 64 ( the default ) which gives you 64 items at a time.

Code

dls.valid.show_batch(max_n=4, nrows=1)

Data Augmentation

Data augmentation refers to creating random variations of the input data such that it appears different without actually changing the fundmental meaning of the data for example adding flipping, perspective warping, brigtness and contrast variations.

Fastai provides us with an aug_transforms function which provides us with a set of standard augmentations that have been found to work pretty well.

We apply these transforms to the entire batch using the batch_tfms parameter.

Code

antelopes = antelopes.new(item_tfms=Resize(128), batch_tfms=aug_transforms())
dls = antelopes.dataloaders(path)
dls.train.show_batch(max_n=8, nrows=2, unique=True)

Code

antelopes = antelopes.new(
    item_tfms=RandomResizedCrop(224, min_scale=0.5),
    batch_tfms=aug_transforms())
dls = antelopes.dataloaders(path)

Now that we have assembled our data in a format fit for model training, let’s actually train an image classifier using it.

Define learner, Train your model, and use it to clean your data

###Learner

Code

learn = vision_learner(dls, resnet18, metrics=error_rate)

/usr/local/lib/python3.9/dist-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
/usr/local/lib/python3.9/dist-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/resnet18-f37072fd.pth" to /root/.cache/torch/hub/checkpoints/resnet18-f37072fd.pth

Code

learn.lr_find()

SuggestedLRs(valley=0.0010000000474974513)

###Train model In this case, we are fine-tuning a pre-trained resnet model.

Code

learn.fine_tune(20, 0.0010000000474974513 )

epoch	train_loss	valid_loss	error_rate	time
0	3.376515	2.119209	0.572414	00:59

95.00% [19/20 18:03<00:57]

epoch	train_loss	valid_loss	error_rate	time
0	2.558627	1.939187	0.523153	01:00
1	2.337216	1.802150	0.490148	00:58
2	2.186192	1.689499	0.463054	00:56
3	1.944700	1.601973	0.454187	01:00
4	1.765508	1.545366	0.436453	00:56
5	1.650410	1.506233	0.416256	00:55
6	1.496741	1.453143	0.430049	00:59
7	1.396045	1.394779	0.411330	00:56
8	1.287408	1.378575	0.395074	00:55
9	1.233312	1.347817	0.397044	00:56
10	1.167068	1.328824	0.389163	00:58
11	1.072845	1.315839	0.384729	00:55
12	1.017716	1.308248	0.381281	00:56
13	0.936392	1.316091	0.380296	00:56
14	0.903043	1.311959	0.376355	00:56
15	0.900170	1.304930	0.376355	00:54
16	0.864651	1.305291	0.372414	00:58
17	0.846347	1.295033	0.369951	00:54
18	0.840998	1.300189	0.369951	00:56

92.06% [116/126 00:40<00:03 0.8602]

epoch	train_loss	valid_loss	error_rate	time
0	2.558627	1.939187	0.523153	01:00
1	2.337216	1.802150	0.490148	00:58
2	2.186192	1.689499	0.463054	00:56
3	1.944700	1.601973	0.454187	01:00
4	1.765508	1.545366	0.436453	00:56
5	1.650410	1.506233	0.416256	00:55
6	1.496741	1.453143	0.430049	00:59
7	1.396045	1.394779	0.411330	00:56
8	1.287408	1.378575	0.395074	00:55
9	1.233312	1.347817	0.397044	00:56
10	1.167068	1.328824	0.389163	00:58
11	1.072845	1.315839	0.384729	00:55
12	1.017716	1.308248	0.381281	00:56
13	0.936392	1.316091	0.380296	00:56
14	0.903043	1.311959	0.376355	00:56
15	0.900170	1.304930	0.376355	00:54
16	0.864651	1.305291	0.372414	00:58
17	0.846347	1.295033	0.369951	00:54
18	0.840998	1.300189	0.369951	00:56
19	0.856114	1.301778	0.371921	00:54

Code

learn.show_results(max_n = 6, figsize=(7,8))

###Visualize results with Confusion Matrix

Code

interp = ClassificationInterpretation.from_learner(learn)
interp.plot_confusion_matrix(figsize = (7,8))

Fastai allows us to see the results of our model using a confusion matrix.For example if we want to see how many antelopes are being classified correctly vs incorreclty.

The rows represent the actual labels of the antelopes in the dataset plotted against the resulting predictions from the model in the columns.

The values in the diagonal represent the correctly classfied while the off- diagonal values represent the incorrectly classified images.

Top losses

We plot our images based off loss starting from the top loss.

This helps us look at the images with the highest loss and we try to look for patterns to theorize whats causing this. Here we actually get to see the image, its prediction, actual label, loss and probability that the model is correct.

A high loss results from high model confidence in a misclassfied image or low confidence in a correclty classified image. This can result from issues in the dataset, the model or both.

Plot images according to losses helps us when doing data cleaning as shown below

Code

interp.plot_top_losses(4, nrows=2,figsize= (10,8) )

Data cleaning with fastai ImageClassifierCleaner

As unintuitive as it seems, training a model before data cleaning can help us identify where some of the issues in our dataset which speeds up our data cleaning process.

Fastai has a GUI tool that enables to view our input data from either training or validation. Furthermore it enables us to delete, re-label and clean our data in various ways. This GUI also sorts images based off loss enabling us to edit the images giving our model issues first.

Sidebar: One thing i notice is you would have to be a domain expert to know which images are incorrectly labelled or categorized.

Code

#hide_output
#images are ordered by loss/confidence
cleaner = ImageClassifierCleaner(learn)
cleaner

The actual functions that execute above chosen tasks i.e. deleting or re-labelling

Code

#hide
for idx in cleaner.delete(): cleaner.fns[idx].unlink()
for idx,cat in cleaner.change(): shutil.move(str(cleaner.fns[idx]), path/cat)

Export Model

You have to export your model so that you can use the model for inference outside of this session.

We use the export method for this. This saves the model architecture, parameters and the definition of how we created our data loaders which defines the transforms to be performed on the dataset etc.

Code

learn.export('antelopeClassifier.pkl')

Let’s check that the file exists, by using the ls method that fastai adds to Python’s Path class:

Code

path = Path()
path.ls(file_exts='.pkl')

(#1) [Path('antelopeClassifier.pkl')]

East African Antelope Classifier Model Deployment

Turning Your Model into an Online Application

Now we know how to save and export the model, load it and use it for inference. In the same parent folder, we create another .ipynb file where we load our saved model, define a fuction to do inference, and create a web interface for it using gradio.

The model is uploaded and served via Hugging face spaces.

Terminate this session, and navigate to the parent directory, open antelopeInference.ipynb and run the cells

In the case of this particular example, i run the inference in the same notebook for ease of display.

Hugging face Spaces + Gradio

This part can be run outside of this current notebook all you would have to do is load the model.

As mentioned above, to enable people interact with our model outside of Jupyter notebook, we deploy the model to the cloud and build a UI on top of that.

To do the above, we host our model on Hugging face Spaces and we use gradio to build a web interface for the model.

This lets other people interact with the underlying model through a decent UI.

In my experience, this part is just as important as any other parts of the whole lifecylce.

In this example, we deploy a model that classifies East african antelopes into categories and gives a score based on confidence.

Using the model for inference

###Load pickle object

We use the load_learner method to load a the model that was trained and exported as a pickle object above. In this case, we already have our model loaded hence no need to call load_learner in this same session.

Code

#hide
learn_inf = load_learner(path/'antelopeClassifier.pkl')

###Inference Here we are going to search duckduck go and download a picture of a Uganda Kob. We store the image in the images folder. Pass a filename of a sample image to the predict method to do inference on that image.

Code

urls = search_images_ddg('Uganda Kob', max_images=1)
urls[0]

Code

!ls images

ugandaKobA.jpg  ugandaKob.jpg

Code

learn.predict('images/ugandaKobA.jpg')

#we would have used this incase we had used the above load_learner method
#learn_inf.predict('images/grizzly.jpg')

('Uganda Kob',
 TensorBase(20),
 TensorBase([3.1946e-05, 8.2230e-06, 9.9609e-06, 5.3365e-04, 2.2620e-06, 6.6840e-05, 1.6654e-04, 3.6462e-05, 6.4279e-05, 3.5542e-05, 7.2383e-06, 7.2009e-05, 8.2187e-04, 2.3515e-05, 3.0744e-04,
             1.6601e-05, 2.9260e-05, 1.9416e-01, 8.6021e-06, 2.9090e-05, 8.0270e-01, 1.1275e-04, 6.8861e-06, 7.5183e-05, 2.9632e-05, 6.3603e-04, 7.1572e-06]))

This has returned three things: the predicted category in the same format you originally provided (in this case that’s a string), the index of the predicted category, and the probabilities of each category. The last two are based on the order of categories in the vocab of the DataLoaders; that is, the stored list of all possible categories. At inference time, you can access the DataLoaders as an attribute of the Learner:

Code

learn.dls.vocab

['Bates’s pygmy', 'Black fronted duiker', 'Blue duiker', 'Defassa Waterbuck', 'Eland', 'Forest duiker', 'Grant’s Gazelle', 'Greater Kudu', 'Grey duiker', 'Guenther’s dik dik', 'Hartebeest', 'Harvery’s red duiker', 'Impala ', 'Klipspringer', 'Oribi', 'Oryx', 'Peter’s duiker', 'Reedbuck', 'Sitatunga', 'The lesser Kudu', 'Uganda Kob', 'bushbuck', 'nyala', 'roan', 'sable', 'steenbok', 'tsessebe']

##Upload model to Hugging Face Spaces

Hugging Face Spaces is a platform that offers git repositories where we can store and host code for Machine Learning apps and demos. In this example, we take our antelope classifier model and upload it to hugging face spaces server.

Spaces also offers us ways to build a UI ontop of this model using frameworks such as streamlit, gradio or with docker images.

In this example, we shall use gradio to build our interface.

Hugging face requires us to have an account before using spaces.If you have one, log in if not create one.

After logging in, navigate to the spaces main page by following the link above or clicking spaces on the navbar.

Click Create new Space.

Give the space a name, in this example we named it antelopeClassifier. Choose a license, I used apache-2.0 as it has the most basic terms and allows it to be reused without me doing anything.

I choose gradio for SDK as thats what i will be using in this particular example to build my UI.

Make it public to enable sharing.

Then Click Create Space.

Remember how i said spaces is a git repository,After creating the space we are navigated to a new page where spaces gives us instructions on how to clone the repo locally, make edits and add new files.

Since its a git repo, this means we can push back our local changes to spaces after we make any changes.

The initial instructions show us how to do a few things * Clone the git repository. * Create new files i.e. our app.py file * Push and save changes on hugging face.

Hugging face also offers us a web interface where we can do the above all in the browser. This is under the Files and versions tab as shown in figure 2. below

##Build Gradio Interface

Since hugging face spaces is a repo as mentioned above, you can clone it to your local workspace.

I open the terminal on my computer and clone the repo using the following command

git clone https://huggingface.co/spaces/silvaKenpachi/antelopeClassifier

We define a gradio Interface class.This class requires us to specify three particular parameters i.e.,

The function that takes input data and runs it against the model
The expected input data
The expected output components

Additional parameters can also be specified to enable customization of the GUI.

Check out the gradio spaces documentation for more.

Define the categories the antelope is to be classified into

::: {.cell 0=‘e’ 1=‘x’ 2=‘p’ 3=‘o’ 4=‘r’ 5=‘t’}

Code

categories = ('Eland', 'Greater Kudu', 'Hartebeest', 'Oryx', 'Defassa Waterbuck', 'Sitatunga', 'Impala ', 'The lesser Kudu', 'Grant’s Gazelle','Reedbuck','Uganda Kob','Forest duiker','Harvery’s red duiker', 'Blue duiker', 'Peter’s duiker','Black fronted duiker','Grey duiker','Oribi','Klipspringer','Guenther’s')

:::

Define the function used to create the GUI.

::: {.cell 0=‘e’ 1=‘x’ 2=‘p’ 3=‘o’ 4=‘r’ 5=‘t’}

Code

def classify_image(img):
  pred,idx,probs = learn.predict(img)
  return dict(zip(categories, map(float,probs)))

:::

Code

im

Code

classify_image(im)

{'Eland': 3.194607052137144e-05,
 'Greater Kudu': 8.22300080471905e-06,
 'Hartebeest': 9.96089329419192e-06,
 'Oryx': 0.000533652026206255,
 'Defassa Waterbuck': 2.262045882162056e-06,
 'Sitatunga': 6.683968240395188e-05,
 'Impala ': 0.00016654374485369772,
 'The lesser Kudu': 3.646174445748329e-05,
 'Grant’s Gazelle': 6.427891639759764e-05,
 'Reedbuck': 3.554236172931269e-05,
 'Uganda Kob': 7.23827042747871e-06,
 'Forest duiker': 7.200949767138809e-05,
 'Harvery’s red duiker': 0.0008218654547818005,
 'Blue duiker': 2.351530929445289e-05,
 'Peter’s duiker': 0.0003074379055760801,
 'Black fronted duiker': 1.6601370589341968e-05,
 'Grey duiker': 2.9260476367198862e-05,
 'Oribi': 0.19416256248950958,
 'Klipspringer': 8.60212094266899e-06,
 'Guenther’s': 2.908950045821257e-05}

Code

urls = search_images_ddg('Impala Antelope', max_images=1)
urls[0]

'https://vignette.wikia.nocookie.net/creatures-of-the-world/images/2/2c/Imp_bill2.jpg/revision/latest?cb=20160423172708'

Code

urls = search_images_ddg('Eland Antelope', max_images=1)
urls[0]

'https://media.istockphoto.com/photos/common-eland-is-the-largest-of-the-african-antelope-species-picture-id1252840135'

::: {.cell 0=‘e’ 1=‘x’ 2=‘p’ 3=‘o’ 4=‘r’ 5=‘t’ outputId=‘1fe225b5-3498-47db-d865-feb515cb20e8’}

Code

#create gradio interface
image = gr.inputs.Image(shape=(128,128))
label = gr.outputs.Label()
examples = ['images/UgandaKob.jpg', 'images/impala.jpg', 'images/eland.jpg']

intf = gr.Interface(fn=classify_image, inputs=image, outputs=label, examples=examples )
intf.launch(inline=False)

/usr/local/lib/python3.9/dist-packages/gradio/deprecation.py:40: UserWarning: `optional` parameter is deprecated, and it has no effect
  warnings.warn(value)
/usr/local/lib/python3.9/dist-packages/gradio/deprecation.py:40: UserWarning: The 'type' parameter has been deprecated. Use the Number component instead.
  warnings.warn(value)

IMPORTANT: You are using gradio version 3.0, however version 3.14.0 is available, please upgrade.
--------
Colab notebook detected. To show errors in colab notebook, set debug=True in launch()
Running on public URL: https://b95a96f1615bcb2a.gradio.app

This share link expires in 72 hours. For free permanent hosting, check out Spaces (https://www.huggingface.co/spaces)

(<gradio.routes.App at 0x7f7aa7230fd0>,
 'http://127.0.0.1:7862/',
 'https://b95a96f1615bcb2a.gradio.app')

:::

In the above function, we define the input data and augmentations. we store this in the image variable shown above.

We define the output as a label. We also provide examples of what our input data looks like.

We then define the gradio interface. We pass in the classify_image fuction which runs a model on the input data and returns results of the category, confidence score expresssed as a percentage. We pass in the expected outputs and examples.

We then run the .launch which launches our demo bear classifier app and provides a local Url.

###Export code with NBDev

Export Hugging face spaces expects a python script We use a library called nbdev to export all the required functions which is what shall go in our initial app.py file.

Download the .py file, and add that file into our cloned local repository for our app.

Code

import nbdev
#nbdev.export.nb_export('drive/MyDrive/ColabNotebooks/app.ipynb', 'drive/MyDrive/ColabNotebooks/lesson2')
nbdev.export.nb_export('gdrive/MyDrive/AntelopeColabNotebooks/antelopeInference.ipynb', 'gdrive/MyDrive/AntelopeColabNotebooks')
print('Export successful')

Export successful

Potential Issues

In the image i took of my phone, i can actually see what appears to be more than one class of antelopes. Hence since an image is classifying the image as a whole, results might be based of one class only.

#Acknowledgments Thanks to Jeremy Howard, Tanishq, Ben Coman, Rachel Thomas and the fast.ai community at large for inspiring me to create this content through their various works such as

If you found this interesting, Please check out more of my work on my personal website

Other Formats

Antelope Classification Application - Introduction

Imports

Data collection

Define datablock and build model

Data Augmentation

Define learner, Train your model, and use it to clean your data

Top losses

Data cleaning with fastai ImageClassifierCleaner

Export Model

East African Antelope Classifier Model Deployment

Turning Your Model into an Online Application

Hugging face Spaces + Gradio

Using the model for inference

Potential Issues