Adding the swish activation function to your Keras model

If you want to know more about the swish activation function, I can highly recommend this blog post and the paper it is based on.

Unfortunately, the Keras code given in the blog post above didn’t work for me but after a while I found the solution somewhere else. So, this is how you can use the swish activation function in Keras:

from keras import backend as K
from keras.utils.generic_utils import get_custom_objects

# needs to be defined as activation class otherwise error
# AttributeError: 'Activation' object has no attribute '__name__'    
class Swish(Activation):
    
    def __init__(self, activation, **kwargs):
        super(Swish, self).__init__(activation, **kwargs)
        self.__name__ = 'swish'

def swish(x):
    return (K.sigmoid(x) * x)

get_custom_objects().update({'swish': Swish(swish)})

Afterwards, just make sure to replace all activation function calls (such as “relu” for example) with “swish”.

Dealing with a “ResourceExhaustedError” in Keras

I’m using Keras (version 2.2.0) with TensorFlow (version 1.8.0) to train ANNs with many input neurons (204800 to be precise) and a rather small database.

I recently started to get the following error message every once in a while:

> ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[204800,100] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc

> [[Node: dense_58/kernel/Assign = Assign[T=DT_FLOAT, use_locking=true, validate_shape=true, _device=”/job:localhost/replica:0/task:0/device:GPU:0″](dense_58/kernel, dense_58/random_uniform)]]

> Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

If you run into the same issue, there are two things to try:

1.) Add the following to your code:

from keras import backend as K
[…]
K.clear_session()

2.) Try to save each model to the hard-drive (resulting in roughly 240 MB for each model in my case) and delete it to free memory for a new model in each iteration. You can delete a model by simply telling Python to free the memory and clean up:

import gc
[…]
del model
gc.collect()

Making a subplot span several columns with pyplot

Currently, I’m using PyPlot a lot. There was one issue bugging me for a few days now and this issue was the simply question how on earth a subplot spanning several columns could look like. There doesn’t seem to be any official documentation mentioning this and even the corresponding Stackoverflow article could only give some hints in the right direction. Finally I found the solution using trial and error:


import matplotlib.pyplot as plt

plt.ion()
fig = plt.figure(figsize=(height_in,width_in), dpi=80)
ax1 = plt.subplot(221)
ax2 = plt.subplot(2,2,(3,4))
ax3 = plt.subplot(222)

In case you are wondering about the meaning of those strange numbers, I quote from the article cited above:

“These are subplot grid parameters encoded as a single integer. For example, “111” means “1×1 grid, first subplot” and “234” means “2×3 grid, 4th subplot”.

Alternative form for add_subplot(111) is add_subplot(1, 1, 1).”

Fixing a broken Matlab installation after an Ubuntu update

Today, I updated Ubuntu as usual and suddenly Matlab stopped working. Instead of starting up nicely, it only complained about some “segmentation violation”. Luckily however, I came across a rather simple solution to this problem. All I had to do was to install the “matlab-support” and “matlab-support-dev” packages via:

sudo aptitude install matlab-support
sudo aptitude install matlab-support-dev

and after that, Matlab worked like a charm again. I just wanted to share that insight in case anyone stumbles across the same problem one day. 🙂

Implementing cross correlation in Matlab without using xcorr

If you’ve ever wanted to implement the cross correlation function in Matlab without using the inbuilt xcorr function, this post might be of some help for you.

This is how such a function could look like:

%Implementation of the cross correlation function
function [result] = crossCorrelation(signalA,signalB)
  signalALength = size(signalA,1);
  signalBLength = size(signalB,1);

  result = zeros(1, signalALength + signalBLength - 1 );
  resultLength = size(result,2);

  for m = 1 : resultLength
    arg = (m - signalALength);

    if(arg < 0)
      negativeCondition = 1;
      limit = signalALength + arg;
    else
      negativeCondition = 0;
      limit = signalALength - arg;
    end

    for n = 1:limit
      if (negativeCondition)
        result(m) = result(m) + signalA(n) * signalB(n - arg);
      else
        result(m) = result(m) + signalA(arg + n) * signalB(n);
      end
    end
  end
end

 

 

 

Fiji Volume Viewer macro extension

While working on a project, I encountered the problem of constructing a 3D volume reconstruction of an image sequence obtained from electron tomography.

The image sequence was given as a stack of .tif files. After some searching on the web I found a very useful tool called Fiji. This tool provides a rich variety of plugins and extensions. It took me some time to understand that the Volumer Viewer (written by Kai Uwe Barthel) plugin and its macro interface Volumer Slicer (written by Michael Doube) might actually be useful for my task as I wanted to record a video of the resulting volume.

Unfortunately, Volume Slicer doesn’t provide the possibility to save and / or export the resulting volume. Thus, I decided to give Volume Slicer a try but its code has been last touched in 2007, which makes it heavily outdated. In order to provide a working environment again I removed some deprecated parts and added some new parameters to the macro.

The code can be found in this gitlab repository: https://gitlab.com/lukasbrausch/volume_slicer_new/tree/master

In case you want to use it with Fiji, all you have to do is to save the code as “something.txt” and create a new macro with this code. Details about Fiji macro programming can be found here.