Adding the swish activation function to your Keras model

lukasbrausch Code, Keras, Python

If you want to know more about the swish activation function, I can highly recommend this blog post and the paper it is based on.

Unfortunately, the Keras code given in the blog post above didn’t work for me but after a while I found the solution somewhere else. So, this is how you can use the swish activation function in Keras:

from keras import backend as K
from keras.utils.generic_utils import get_custom_objects

# needs to be defined as activation class otherwise error
# AttributeError: 'Activation' object has no attribute '__name__'    
class Swish(Activation):
    
    def __init__(self, activation, **kwargs):
        super(Swish, self).__init__(activation, **kwargs)
        self.__name__ = 'swish'

def swish(x):
    return (K.sigmoid(x) * x)

get_custom_objects().update({'swish': Swish(swish)})

Afterwards, just make sure to replace all activation function calls (such as “relu” for example) with “swish”.

Installing Brave Browser on Debian 9

lukasbrausch Debian 
curl -s https://brave-browser-apt-release.s3.brave.com/brave-core.asc | sudo apt-key --keyring /etc/apt/trusted.gpg.d/brave-browser-release.gpg add -

source /etc/os-release

echo "deb [arch=amd64] https://brave-browser-apt-release.s3.brave.com/ bionic main" | sudo tee /etc/apt/sources.list.d/brave-browser-release-boinic.list

sudo apt update

sudo apt install brave-keyring brave-browser

Dealing with a “ResourceExhaustedError” in Keras

lukasbrausch Code, Keras, Python

I’m using Keras (version 2.2.0) with TensorFlow (version 1.8.0) to train ANNs with many input neurons (204800 to be precise) and a rather small database.

I recently started to get the following error message every once in a while:

> ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[204800,100] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc

> [[Node: dense_58/kernel/Assign = Assign[T=DT_FLOAT, use_locking=true, validate_shape=true, _device=”/job:localhost/replica:0/task:0/device:GPU:0″](dense_58/kernel, dense_58/random_uniform)]]

> Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

If you run into the same issue, there are two things to try:

1.) Add the following to your code:

from keras import backend as K
[…]
K.clear_session()

2.) Try to save each model to the hard-drive (resulting in roughly 240 MB for each model in my case) and delete it to free memory for a new model in each iteration. You can delete a model by simply telling Python to free the memory and clean up:

import gc
[…]
del model
gc.collect()

Making a subplot span several columns with pyplot

lukasbrausch Code, Python

Currently, I’m using PyPlot a lot. There was one issue bugging me for a few days now and this issue was the simply question how on earth a subplot spanning several columns could look like. There doesn’t seem to be any official documentation mentioning this and even the corresponding Stackoverflow article could only give some hints in the right direction. Finally I found the solution using trial and error:


import matplotlib.pyplot as plt

plt.ion()
fig = plt.figure(figsize=(height_in,width_in), dpi=80)
ax1 = plt.subplot(221)
ax2 = plt.subplot(2,2,(3,4))
ax3 = plt.subplot(222)

In case you are wondering about the meaning of those strange numbers, I quote from the article cited above:

“These are subplot grid parameters encoded as a single integer. For example, “111” means “1×1 grid, first subplot” and “234” means “2×3 grid, 4th subplot”.

Alternative form for add_subplot(111) is add_subplot(1, 1, 1).”

Fixing a broken Matlab installation after an Ubuntu update

lukasbrausch Matlab, Ubuntu

Today, I updated Ubuntu as usual and suddenly Matlab stopped working. Instead of starting up nicely, it only complained about some “segmentation violation”. Luckily however, I came across a rather simple solution to this problem. All I had to do was to install the “matlab-support” and “matlab-support-dev” packages via:

sudo aptitude install matlab-support
sudo aptitude install matlab-support-dev

and after that, Matlab worked like a charm again. I just wanted to share that insight in case anyone stumbles across the same problem one day. 🙂

Implementing cross correlation in Matlab without using xcorr

lukasbrausch Code, Matlab

If you’ve ever wanted to implement the cross correlation function in Matlab without using the inbuilt xcorr function, this post might be of some help for you.

This is how such a function could look like:

%Implementation of the cross correlation function
function [result] = crossCorrelation(signalA,signalB)
  signalALength = size(signalA,1);
  signalBLength = size(signalB,1);

  result = zeros(1, signalALength + signalBLength - 1 );
  resultLength = size(result,2);

  for m = 1 : resultLength
    arg = (m - signalALength);

    if(arg < 0)
      negativeCondition = 1;
      limit = signalALength + arg;
    else
      negativeCondition = 0;
      limit = signalALength - arg;
    end

    for n = 1:limit
      if (negativeCondition)
        result(m) = result(m) + signalA(n) * signalB(n - arg);
      else
        result(m) = result(m) + signalA(arg + n) * signalB(n);
      end
    end
  end
end

 

Â