.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "gallery_examples\multiscaleMNIST.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_gallery_examples_multiscaleMNIST.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_gallery_examples_multiscaleMNIST.py:


Multiscale MNIST Examples
==========================

We illustrate the class :class:`codpy.multiscale_kernel.MultiScaleKernel`, applying it to the `MNIST <https://en.wikipedia.org/wiki/MNIST_database>`_ problem.
The methodology is similar to the gallery example :ref:`MNIST <MNIST>`_

.. GENERATED FROM PYTHON SOURCE LINES 8-24

.. code-block:: Python


    import os
    import pandas as pd
    import numpy as np
    import random
    # We use a custom hot encoder for performances reasons.
    from codpy.data_processing import hot_encoder
    # Standard codpy kernel class.
    from codpy.kernel import Kernel
    # A multi scale kernel method.
    from codpy.multiscale_kernel import *
    from sklearn.metrics import confusion_matrix

    os.environ["OPENBLAS_NUM_THREADS"] = "32"
    os.environ["OMP_NUM_THREADS"] = "32"








.. GENERATED FROM PYTHON SOURCE LINES 25-48

.. code-block:: Python

    def get_MNIST_data(N=-1):
        import tensorflow as tf

        (x, fx), (z, fz) = tf.keras.datasets.mnist.load_data()
        x, z = x / 255.0, z / 255.0
        x, z, fx, fz = (
            x.reshape(len(x), -1),
            z.reshape(len(z), -1),
            fx.reshape(len(fx), -1),
            fz.reshape(len(fz), -1),
        )
        fx, fz = (
            hot_encoder(pd.DataFrame(data=fx), cat_cols_include=[0], sort_columns=True),
            hot_encoder(pd.DataFrame(data=fz), cat_cols_include=[0], sort_columns=True),
        )
        x, fx, z, fz = (x, fx.values, z, fz.values)
        if N != -1:
            indices = random.sample(range(x.shape[0]), N)
            x, fx = x[indices], fx[indices]

        return x, fx, z, fz









.. GENERATED FROM PYTHON SOURCE LINES 49-60

.. code-block:: Python

    def show_confusion_matrix(z, fz, predictor=None, cm=True):
        f_z = predictor(z)
        fz, f_z = fz.argmax(axis=-1), f_z.argmax(axis=-1)
        out = confusion_matrix(fz, f_z)
        if cm:
            print("confusion matrix:")
            print(out)
        print("score MNIST:", np.trace(out) / np.sum(out))
        pass









.. GENERATED FROM PYTHON SOURCE LINES 61-62

Run codpy silently on/off.

.. GENERATED FROM PYTHON SOURCE LINES 62-64

.. code-block:: Python

    core.kernel_interface.set_verbose(False)








.. GENERATED FROM PYTHON SOURCE LINES 65-66

The training set is `x,fx`, the test set is `z,fz`.

.. GENERATED FROM PYTHON SOURCE LINES 66-70

.. code-block:: Python

    N_clusters=100
    N_MNIST_pics=5000
    x, fx, z, fz = get_MNIST_data(N_MNIST_pics)








.. GENERATED FROM PYTHON SOURCE LINES 71-72

Select a multi scale kernel method where the centers are given by a k-mean algorithm.

.. GENERATED FROM PYTHON SOURCE LINES 72-80

.. code-block:: Python

    N_partition=5
    predictor = MultiScaleKernelClassifier(x=x,fx=fx,N=N_partition,method=MiniBatchkmeans)
    print("Reproductibility test:")
    show_confusion_matrix(x, fx, predictor,cm=False)
    print("Performance test:")
    show_confusion_matrix(z, fz, predictor,cm=False)






.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Reproductibility test:
    score MNIST: 1.0
    Performance test:
    score MNIST: 0.9604




.. GENERATED FROM PYTHON SOURCE LINES 81-82

Select a multi scale kernel where the centers are given by a greedy search algorithm.

.. GENERATED FROM PYTHON SOURCE LINES 82-87

.. code-block:: Python

    predictor = MultiScaleKernelClassifier(x=x,fx=fx,N=N_partition,method=GreedySearch)
    print("Reproductibility test:")
    show_confusion_matrix(x, fx, predictor,cm=False)
    print("Performance test:")
    show_confusion_matrix(z, fz, predictor,cm=False)




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Reproductibility test:
    score MNIST: 1.0
    Performance test:
    score MNIST: 0.9604





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (1 minutes 51.665 seconds)


.. _sphx_glr_download_gallery_examples_multiscaleMNIST.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: multiscaleMNIST.ipynb <multiscaleMNIST.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: multiscaleMNIST.py <multiscaleMNIST.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: multiscaleMNIST.zip <multiscaleMNIST.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_