\( \newcommand{\matr}[1] {\mathbf{#1}} \newcommand{\vertbar} {\rule[-1ex]{0.5pt}{2.5ex}} \newcommand{\horzbar} {\rule[.5ex]{2.5ex}{0.5pt}} \newcommand{\E} {\mathrm{E}} \)
deepdream of a sidewalk

Color

I'm interested in how people experience color. These pages are a work in progress. As I write this, I'm hoping to try and begin to form an answer to some of the questions raised below.

In the space that the human brain builds to represent the surrounding world, the color that the brain assigns to a surface of an object in this space is dependent on many things in addition to the light coming from the corresponding surface in the real world. This is a known phenomenon, and one of the reasons people developed the field of color appearance models. With this in mind, it is interesting to ask: do modern neural networks for image recognition integrate the same information to identify certain colors? If not, there are likely conditions under which they perform poorly as a result.

The above question can be played in reverse: if neural networks do model colors well, then will inspecting the internals of a trained network help reveal how color might be processed in human brains?

Experiments below are exported from Jupyter Lab notebooks from this project's git repository.

Experiment 1. Summary.

Overview of progress in experiment 1.

Experiment 2. Summary.

Overview of progress in experiment 2.

Experiment 1.1.1

Creating an orange vs. brown dataset.

Experiment 1.1.2

This is the second attempt at experiment 1.1. Same setup, more data collected.

Experiment 1.2.1

Looking for ImageNet classes that differ only in color.

Experiment 1.3

Color dot dataset.

Experiment 1.3.1

ResNet transfer learning for the orange-brown-neither dataset.

Experiment 1.3.2

Repeat 1.3.1 (fixing a mistake of 1.3.1)

Experiment 1.4.1

Testing orange and brown at the final pool layer.

Experiment 1.5.1

Experiment 1.3, but with the single circle being varied by size and position.

Experiment 1.6.1

A green-blue version of 1.5, carried out for the purposes of comparison.

Experiment 2.1.1

Searching for a image classification fail case by varying illumination.

Experiment 2.1.2

Searching for an image classification fail case by varying illumination (2nd attempt).

Experiment 2.2.1

Testing to see if a simple high pass filter can be used to improve classification accuracy in the presence of atypical illumination.

Concept

This page is a scrapbook of ideas for the project. Color and neural networks I’m hoping I can stumble on a better description of human color perception by trying to answer the question: What can be said about the relationship between neural networks for vision and human color perception? In many ways, this question has issues. The topic of human color perception is the subject of deep unanswered questions. Color is a property of our mind, an appearance in consciousness and I am currently of the belief that we won’t get a satisfying answer to the question what is color? Read more...

Refectances of various objects

This webpage was created to investigate the reflectance data collected by Morimoto et al. as part of their 2022 publication “Spectral measurement of daylights and surface properties of natural objects in Japan”. The dataset is made available at https://zenodo.org/record/5189606. While the data was published in 2022, the data was collected much earlier in 2013 and 2014.