Some Useful pytest Command-line Options

I love pytest.

Pytest is a testing framework which allows us to write test codes using functional python and functional python is awesome.

Why use PyTest?

There are many reasons to use pytest here are some that I feel are important.

• Very easy to start with because of its simple and easy syntax.
• Less Boilerplate
• Can run a specific test or a subset of tests
• and many more useful features

Here’s a list of command-line options that can be used while using pytest.

Simple use
pytest

Too unorganised, lets’ fix this

pytest -v

Much better.

oh there’s a failure but there is too much information on the failure, let’s fix that with

pytest -v –tb=line

This is good, but just a line of info is too little. Ok lets try this.

pytest -v –tb=short

Thats good.

What if I want to run a specific test. No problem just use “-k” option

pytest -v -k “SOMENAME”

Thats cool. What if I want to just run the last failed test or tests. Simple use “–lf“

pytest -v –lf

And if you want to debug the failed tests, well use “–pdb“

pytest -v –pdb

On failure, it will bring the debugger.

Well, that’s it for this post. Hope this helps.

More posts like this that you might want to explore.

Advertisement

How to Use Git Format Patch

Just like python, I love using git. It’s versatile and does simplify a lot of developmental tasks.

With a distributed version control system like Git, we can actually send a path for someone to review without the need to have the centralized git repository using the git format-patch feature.

This is useful when the remote git repository is down or behind or behind an artificial throttling IT system.

Here are few ways to create patches

Creating patches from commits

let’s say you are in the “bug_fix“ branch and you want to have all the patches since master.

git format-patch master

This will create multiple patches per commit. We can create a single patch for multiple commits by using the –stdout option and directing the output to a file:

git format-patch master --stdout > mypatch.patch

One can also create a patch from git diff

git diff > something.patch

or from cache

git diff --cached > something.patch

How to use the patch?

Once you have received a patch how do you use it?

When you receive a patch file from someone, you can easily apply the patch using the git am command or git apply command.

git apply mypatch.patch

This command applies the patch but does not create a commit.

git am mypathc.patch

This command applies the patch and does create a commit with the original command

Hope this helps. It really helps me these days.

Deploying Django App with PythonAnywhere

Recently wanted to deploy a learning app I have developed for my kids on the web. The main reason behind this was to make the app accessible from anywhere and from any device.

This was necessitated by the our travel back to kids grandparents, where the laptop was not always online and on.

After research ( few google searches and blog reads…:) ) heroku and pythonanywhere.com were the two method that I settled on.

I have previous use heroku to deploy one of my previous app, so this time turned to pythonanywhere.com Heroku has ton of tutorials on how to deploy a python django app on heroku, but there were not many for pythonanywehere.com, so here’s a pictorial demo of the process I took.

Here’s an overview of the steps involved.

1. Create an pythonanywhere account
2. Upload your code to PythonAnywhere
3. Set up a virtualenv and install Django and any other requirements
4. Set up your web app using the manual config option
5. Add any other setup (static files, environment variables etc)

Reading/Writing wav files with Python

Problem:

You want to read and manipulate sound (*.wav) files but since you are on a company-controlled system, you are not allowed to install pyaudio or other excellent audio packages.

How do you solve this using basic standard python package?

Solution:

from scipy.io.wavfile import read, write

Example:

from scipy.io.wavfile import read, write

fname="some.wav"
rate, data = read(fname)

# data is a numpy array which can be manipulated as needed.
# here  multiplying it 20 time
data = data*20
# let's write it out

write("output.wav", rate, data)

These two functions recently helped me in a personal machine learning project. I love it when such simple solutions are available just out of the box.

Help on the two functions.

Signature: read(filename, mmap=False)
Docstring:
Open a WAV file

Return the sample rate (in samples/sec) and data from a WAV file.

Parameters
----------
filename : string or open file handle
    Input wav file.
mmap : bool, optional
    Whether to read data as memory-mapped.
    Only to be used on real files (Default: False).

    .. versionadded:: 0.12.0

Returns
-------
rate : int
    Sample rate of wav file.
data : numpy array
    Data read from wav file.  Data-type is determined from the file;

Signature: write(filename, rate, data)

Docstring:
Write a numpy array as a WAV file.

Parameters
----------
filename : string or open file handle
    Output wav file.
rate : int
    The sample rate (in samples/sec).
data : ndarray
    A 1-D or 2-D numpy array of either integer or float data-type.

Notes
-----
* Writes a simple uncompressed WAV file.
* To write multiple-channels, use a 2-D array of shape
  (Nsamples, Nchannels).
* The bits-per-sample and PCM/float will be determined by the data-type.

Installing UQTkV3.0 on MAC OS x 10

Version 3 of UQ Toolkit is out.

The UQ Toolkit (UQTk) is a collection of libraries and tools for the quantification of uncertainty in numerical model predictions. Just like version 2, version 3.0 offers intrusive and non-intrusive methods for propagating input uncertainties through computational models, tools for sensitivity analysis, methods for sparse surrogate construction, and Bayesian inference tools for inferring parameters from experimental data.

Installing UQTk version 2 on a Mac was easy. Use gcc or clang and fire up the make file, but for version 3, the team has now shifted to cmake and if you want python interface you also need SWIG.

Just like in life, additional dependencies bring additional complexity and chances of things going wrong. Well this was my experience while installing UQtk version 3 on my mac.

Here are my running notes while installing the library on OS X 10.

Cmake doesn’t come preinstalled on a Mac. So get it from here. Installation is easy.

I didn’t have swig so had to download swig from here

Installing is 4 steps

• Go to the untarred directory,
• ./configure
• make
• sudo make install

I forgot the 4th step and got the following error while installing uqtk

CMake Error at /Applications/CMake.app/Contents/share/cmake-3.6/Modules/FindPackageHandleStandardArgs.cmake:148(message):

Could NOT find SWIG (missing: SWIG_EXECUTABLE SWIG_DIR)

Call Stack (most recent call first): /Applications/CMake.app/Contents/share/cmake-3.6/Modules/

— Configuring incomplete, errors occurred!

See also “/Users/sukhbindersingh/PROJECTS/UQTk_v3.0/builddir/CMakeFiles/CMakeOutput.log”.

 

So make sure, you do all the four steps…

 

SWIG needs PCRE-Perl compatible Regular expression library, which was not available on my mac, so download that too from here

Download PCRE not PCRE2. I wasted time installing PCRE2 to find that swig doesn’t recognise it.

 

Make CMAKE work in command line

 

Now cmake opens up as GUI in Mac OS by defauly, but we need command line version, so we have following 3 options

One may add CMake to the PATH:

PATH=”/Applications/CMake.app/Contents/bin”:”$PATH”

Or, to install symlinks to ‘/usr/local/bin’, run:

sudo “/Applications/CMake.app/Contents/bin/cmake-gui” –install

Or, to install symlinks to another directory, run:

sudo “/Applications/CMake.app/Contents/bin/cmake-gui” –install=/path/to/bin

 

I opted for the second option, mostly because it was simplest and I want cmake to available everywhere and every time I need it

sudo “/Applications/CMake.app/Contents/bin/cmake-gui” –install

 

Where are your gcc, gfortran, and clang?

 

Once we have all these and before you start, we have to check few things. In this installation, I wanted to use gcc, gfortran, and clang for my setup, so get to know where they are located in your system by using the ‘which’ command

The Installations

 

In the terminal, go to the UQTk_v3.0 directory

• mkdir builddir
  

• cd builddir/
  

• cmake -DCMAKE_Fortran_COMPILER=/usr/local/bin/gfortran -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_CXX_COMPILER=/usr/bin/clang ../../UQTk_v3.0
  

 

Replace green text with your own path.

Once the cmake configuration is complete successfully, type make

All will go ok, until you hit this error.

In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:23:

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/

MacOSX10.11.sdk/usr/include/unistd.h:510:14:error:declaration of ‘optarg’ has a different language linkage

extern char *optarg; /*

getopt(3) external variables */

^

/Users/sukhbindersingh/PROJECTS/UQTk_v3.0/cpp/lib/include/gen_defs.h:49:14: note:

previous declaration is here

extern char *optarg;

^

1 error generated.

make[2]: *** [cpp/lib/CMakeFiles/uqtk.dir/tools/multiindex.cpp.o] Error 1

 

Well this is easy to solve and I think it’s an error in the file. Open gen_defs.h file from cpp/lib/include and comment the following line as shown.

extern char *optarg

 

 

Another error that I faced was the missing gfortran library because of no LIBRARY_PATH.

ld: library not found for -lgfortran

clang: error: linker command failed with exit code 1 (use -v to see invocation)

make[2]: *** [cpp/app/gen_mi/gen_mi] Error 1

make[1]: *** [cpp/app/gen_mi/CMakeFiles/

gen_mi.dir/all] Error 2

make: *** [all] Error 2

 

This meant my libgfortran.a was not in the LIBRARY_PATH, so let’s find where the library is and then set the LIBRARY_PATH environment variable pointing it to that folder.

Locate libgfortran*.a

sudo find /usr -iname ‘libgfortran*.a’

 

Found them here.

/usr/local/gfortran/lib/i386/libgfortran.a

/usr/local/gfortran/lib/libgfortran.a

Set the environment variable to that folder.

export LIBRARY_PATH=/usr/local/gfortran/lib/

 

Compilation should continue smoothly and at last type make install and you should have everything installed.

Type ctest to check your installation

Recap

 

So here’s recap of the steps.

1. mkdir builddir
2. cd builddir
3. cmake -DCMAKE_Fortran_COMPILER=/usr/local/bin/gfortran -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_CXX_COMPILER=/usr/bin/clang ../../UQTk_v3.0 [Without python]

   or cmake -DPyUQTk=ON -P-DCMAKE_Fortran_COMPILER=/usr/local/bin/gfortran -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_CXX_COMPILER=/usr/bin/clang ../../UQTk_v3.0 [With python]

4. make
5. make install

I faced two problems,

1. error:declaration of ‘optarg’ has a different language linkage Solution: comment the offending line.
2. ld: library not found for –lgfortran clang: error: linker command failed with exit code 1 Solution: export LIBRARY_PATH=/usr/local/gfortran/lib/

Using PyUQTk option gave me lot of problem, have delayed installing python binding for now due to lack of time.

Well now this is done, will spend couple of weekends looking at the library, you can fork the entire library from my github page. It has all the libs, so if the compliers are similar, they can be used as is without going through all this steps.

Using Aliases In windows CMD

If you are like me and use mac and Linux at home and use windows at work. Then the following will be regular errors.

‘ls’ is not recognized as an internal or external command, operable program or batch file.

I always use ls, clear, cat etc. in my CMD shell and eventually get the following errors. This has happened a lot of time for me to do something about it. Fortunately, there’s an easy way to setup alias in the cmd windows and never see this nasty ‘not recognised’ error. First create a text file like this.

You can use any other commands that you use regularly. Next, open the cmd windows and type the following command

Doskey /macrofile=path/to/alias/file/filename

As shown in the next figure. That’s it.

Here’s how it works, no more nag error messages

Don’t want to type this message every time, create a windows shortcut like this  

That’s it!!

FEM in Python A Simple Start Guide

Wrote this a couple of months back. Yet another tutorial in python, if you are interested in finite element analysis. Nicely goes with this excellent tutorial on FEM.

Click the below link to view the tutorial. FEM with Python

Exploring Matplotlib Styles

Last week got some free time. Used it to upgrade my python installation on Mac, a long awaited task.

Looking at the upgrade log, was most excited to finally see the new version of matplotlib.

So launched it and went straight to the new style package.

Matplotlib is great at graphs but the default style before 1.4.3 left many things wanting.

The style package adds support for easy-to-switch plotting “styles” with the same parameters as a matplotlibrc file.

import matplotlib.pyplot as plt

What are different styles available in matplotlib?

print(plt.style.available)

[u'dark_background', u'bmh', u'grayscale', u'ggplot', u'fivethirtyeight']

Here’s how to use this.

But first let’s generate some data

import numpy as np
data = np.sin(np.linspace(0, 2*np.pi))

The default plot

plt.plot(data, 'r-o')

Let’s use ggplot

plt.style.use('ggplot') 
plt.plot(data, 'r-o')

Dark Background like excel 2007

plt.style.use('dark_background')
plt.plot(data, 'r-o')

BMH style

plt.style.use(‘bmh’)
plt.plot(data, 'r-o')

Graystyle

plt.style.use(‘grayscale’)
plt.plot(data, 'r-o')

fivethirtyeight Style

plt.style.use(‘fivethirtyeight’)
plt.plot(data, 'r-o')

We can even add our own custom .mplstyle files to ~/.matplotlib/stylelib or call use with a URL pointing to a file with matplotlibrc settings. Follow the following link to define your own style.

A Gentle Introduction to the Finite Element Method

Almost every engineer today have some knowledge of theoretical and practical aspects of FEM. And many have played with various software packages at some point of time.

Well that is enough if you just want to perform analysis but if you want to truly understand the mathematical aspects of FEM and want to get a feel of the numerical methods under the hood of FEM then this short lecture is for you.

Here’s the introduction, plucked straight from the course. IF you like it, you will enjoy the lecture.

The course is divided into five lessons and all this in 100 pages. Happy Learning!!

If you haven’t been hiding under a stone during your studies of engineering, mathematics or physics, it is very likely that you have already heard about the Finite Element Method. Maybe you even know some theoretical and practical aspects and have played a bit with some FEM software package. What you are going to find here is a detailed and mathematically biased introduction to several aspects of the Finite Element Method.

This is not however a course on the Analysis of the method. It is just a demonstration of how it works, written as applied mathematicians usually write it. There is going to be mathematics involved, but not lists of theorems and proofs. We are also going from the most particular cases towards useful generalizations, from example to theory.

It is going to be one hundred pages with many figures and many ideas repeated over and over, so that you can read it with ease. These notes have evolved during the decade I have been teaching finite elements to mixed audiences of mathematicians, physicists and engineers. The tone is definitely colloquial. I could just claim that these are my classnotes and that’s what I’m like.

There’s much more than that. First, I believe in doing your best at being entertaining when teaching. At least that’s what I try. Behind that there is
a deeper philosophical point: take your work (and your life) seriously but, please, don’t take yourself too seriously.

I also believe that people should be duly introduced when they meet. All this naming
old time mathematicians and scientists only by their last names looks to me too much
like the Army. Or worse, high school!

I think you have already been properly introduced to the great Leonhard Euler, David Hilbert, Carl Friedrich Gauss, Pierre Simon Laplace and George Green. If you haven’t so far, consider it done here. This is not about history.

It’s just good manners. Do you see what I mean by being colloquial?

Anyway, this is not about having fun, but since we are at it, let us try to have a good time while learning. If you take your time to read these notes with care and try the exercises at the end of each lesson, I can assure that you will have made a significant step in your scientific persona. Enjoy!

Click the below link to access the excellent PDF. Don’t forget to thanks to Francisco Jaview Sayas !

A gentle introduction to the Finite Element Method by Francisco–Javier Sayas

Quick Intro to 2D and 3D Visualization in Python

Posted a quick tutorial introducing visualization in python.

This tutorial offers a quick look at some common 2d and 3d Visualization available in python. This tutorial is meant to give anyone a quick start in matplotlib and mayavi.mlab.

The tutorial was written as part of CSRP program of AeSIAA which I am volunteering as a mentor to few aeronautical students.

View the tutorial by clicking this Quick Intro to 2D and 3D Visualization in Python

Quick Basic Python Tutorial

I have just posted a Quick Basic Python Tutorial on github.

View it here  Quick Introduction to Basic Python.

Short and simple for those who cannot wait to get started.  

LS-DYNA Examples

Well this one will be a short post.

If you are into analysis and by any chance use LS-Dyna analysis software tool, then I have a site to recommend to you all.

It’s www.dynaexamples.com !!

Yes as the name suggest there are numerous downloadable ls-dyna examples on this website.

Even if you are not into Ls-dyna, but are in analysis, I suggest you take a tour of this website. You will come out by learning something useful.

That’s it, if you know about other sites that might be helpful to other users of this blog, please do name it in comments.

Simple Introduction To Working With HDF In PYTHON?

In May 2014 I wrote this tutorial, but stupidly made it as a gif format. Don’t know why?

Today had the chance to revisit that page and I felt the GIF format made the tutorial useless, so dumped it and used ipython notebook to create a more updated but simple tutorial to get started working with HDF in python.

If you are into scientific coding, you will encounter HDF and I hope this simple tutorial will get you started in python. Enjoy!

Click here to view the tutorial

Voronoi with Python

A few weeks back, a colleague was searching for matlab for a task to get the Voronoi diagram from some points he had. I suggested why not use #python? He was not convinced and he got hold of matlab license and did his job.

But I had this itch for trying Voronoi in python, so here it is. Finally got some spare time. It was so easy in python, that I built the example around a tkinter gui.

But before we go to code, for everyone’s benefit What is Voronoi Diagram?

In mathematics, a Voronoi diagram is a way of dividing space into a number of regions. A set of points (called seeds, sites, or generators) is specified beforehand and for each seed there will be a corresponding region consisting of all points closer to that seed than to any other. The regions are called Voronoi cells. It is dual to the Delaunay triangulation.

via wikipedia

And here’s the code.

Rotate

You don’t learn cool things everyday. And today is not everyday, learnt this neat trick to rotate a matrix with a one liner in #python!!!

b = np.array(zip(*a[::-1]))

Simple and powerful. Waiting for a chance to use it somewhere in actual code…

Want to understand what’s happening visit this page by Peter Norvig, director of research Google!!

Python GUI Development Video Tutorial

Found this is at google+… Useful for anyone interested in Python GUI development. Found this to be a very helpful resource for learning to use PySide.

Sit back, watch and learn!!!

update:
if for some reason, the video do not work for you, please visit this youtube link.

Brief Intro to FEA or Finite Element Method (FEM)

Recently someone asked me for a simple and short primer or refresher material on Finite element Method.

Here’s a super simple, brief, to the point , no nonsense PDF on FEA theory from the university of victoria.

Brief and simple. Just enough material to spark interest for the curious!!!

Brief Intro to FEA or Finite Element Method (FEM) [PDF]

Do you know any such good, brief and useful resource? Please share.

3 Simple Steps to fix ipython notebook error on MAC OS X

I thought I did something stupid to my installation of python on mac and python notebook stopped working. ipython works but ipython notebook went haywire.

Tried reinstalling python but the problem remained. Everytime got this error when ipython notebook was invoked.

return _parse_localename(localename)
File "/Users/millerc/anaconda/lib/python2.7/locale.py", line 435, in _parse_localename
raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8

Today, the lazy me corrected the problem with help from this post. It turns out the error occurs after installing the latest Mac OSX 64-bit python.

Fortunately the solution is simple. Follow the 3 simple solution steps

Step 1
open your .bash_profile

Step 2
Add these lines to your .bash_profile:

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

Step 3
Reloaded the profile:

source ~/.bash_profile

Now run ipython notebook

ipython notebook

Done.