The Best Debugging Tool of All

Rob Pike (one of the authors of the Go language) has this to say on debugging…..:

A year or two after I’d joined the Labs, I was pair programming with Ken Thompson on an on-the-fly compiler for a little interactive graphics language designed by Gerard Holzmann. I was the faster typist, so I was at the keyboard and Ken was standing behind me as we programmed. We were working fast, and things broke, often visibly—it was a graphics language, after all. When something went wrong, I’d reflexively start to dig in to the problem, examining stack traces, sticking in print statements, invoking a debugger, and so on. But Ken would just stand and think, ignoring me and the code we’d just written. After a while I noticed a pattern: Ken would often understand the problem before I would, and would suddenly announce, “I know what’s wrong.” He was usually correct. I realized that Ken was building a mental model of the code and when something broke it was an error in the model. By thinking about *how* that problem could happen, he’d intuit where the model was wrong or where our code must not be satisfying the model.

Ken taught me that thinking before debugging is extremely important. If you dive into the bug, you tend to fix the local issue in the code, but if you think about the bug first, how the bug came to be, you often find and correct a higher-level problem in the code that will improve the design and prevent further bugs.

I recognize this is largely a matter of style. Some people insist on line-by-line tool-driven debugging for everything. But I now believe that thinkingwithout looking at the code—is the best debugging tool of all, because it leads to better software.

Advertisements

Updated Binary STL File Reader

bent_plate

A recent comment on the post titled binary stl file reader in numpy prompted me to revisit the code and while I was looking at it , I updated the same for code to view the loaded stl file.

The above picture shows the result.

The code as usual available on github from here

from binarySTLreader import BinarySTL,ShowSTLFile
h,p,n,v1,v2,v3=BinarySTL('bent_plate.stl')
ShowSTLFile(v1,v2,v3)

Switch from Discrete Mathematics to Probability, Statistics

From the book Foundations of Data Science by John Hopcroft and Ravindran Kannan

Computer science as an academic discipline began in the 60’s. Emphasis was on programming languages, compilers, operating systems, and the mathematical theory that supported these areas. Courses in theoretical computer science covered finite automata,regular expressions, context free languages, and computability.

In the 70’s, algorithms was added as an important component of theory. The emphasis was on making computers useful. Today, a fundamental change is taking place and the focus is more on applications.

There are many reasons for this change. The merging of computing and communications has played an important role. The enhanced ability to observe, collect and store data in the natural sciences, in commerce, and in other fields calls for a change in our understanding of data and how to handle it in the modern setting. The emergence of the web and social networks, which are by far the largest such structures, presents both opportunities and challenges for theory

All this entails, there is the switch from discrete mathematics to more emphasis on probability and statistics.

Joint plot with Matplotlib

Today I am releasing a simple module to create joint plot with Matplotlib on github. Joint plot is available in the excellent seaborn library but unfortunately it’s not always available on many systems. Recently I needed this functionality, so wrote this simple module with matplotlib.

The functionality is almost similar to seaborn but with limited feature. This has helped me in my work, releasing it in the hope that others might find it useful.

Sample usage:

Import Jointplot
tips=pd.read_csv(r'tests/tips_p.csv')
data=np.c_[tips['total_bill'].values,tips['tip'].values]
jointPlot(data,kde=True)

Find the code at this github repository.

Installing UQTkV3.0 on MAC OS x 10

Version 3 of UQ Toolkit is out.

The UQ Toolkit (UQTk) is a collection of libraries and tools for the quantification of uncertainty in numerical model predictions. Just like version 2, version 3.0 offers intrusive and non-intrusive methods for propagating input uncertainties through computational models, tools for sensitivity analysis, methods for sparse surrogate construction, and Bayesian inference tools for inferring parameters from experimental data.

Installing UQTk version 2 on a Mac was easy. Use gcc or clang and fire up the make file, but for version 3, the team has now shifted to cmake and if you want python interface you also need SWIG.

Just like in life, additional dependencies bring additional complexity and chances of things going wrong. Well this was my experience while installing UQtk version 3 on my mac.

Here are my running notes while installing the library on OS X 10.

Cmake doesn’t come preinstalled on a Mac. So get it from here. Installation is easy.


I didn’t have swig so had to download swig from here

Installing is 4 steps

  • Go to the untarred directory,
  • ./configure
  • make
  • sudo make install

I forgot the 4th step and got the following error while installing uqtk

CMake Error at /Applications/CMake.app/Contents/share/cmake-3.6/Modules/FindPackageHandleStandardArgs.cmake:148(message):

Could NOT find SWIG (missing: SWIG_EXECUTABLE SWIG_DIR)

Call Stack (most recent call first): /Applications/CMake.app/Contents/share/cmake-3.6/Modules/

— Configuring incomplete, errors occurred!

See also “/Users/sukhbindersingh/PROJECTS/UQTk_v3.0/builddir/CMakeFiles/CMakeOutput.log”.

 

So make sure, you do all the four steps…

 

SWIG needs PCRE-Perl compatible Regular expression library, which was not available on my mac, so download that too from here

Download PCRE not PCRE2. I wasted time installing PCRE2 to find that swig doesn’t recognise it.

 

Make CMAKE work in command line

 

Now cmake opens up as GUI in Mac OS by defauly, but we need command line version, so we have following 3 options

One may add CMake to the PATH:

PATH=”/Applications/CMake.app/Contents/bin”:”$PATH”

Or, to install symlinks to ‘/usr/local/bin’, run:

sudo “/Applications/CMake.app/Contents/bin/cmake-gui” –install

Or, to install symlinks to another directory, run:

sudo “/Applications/CMake.app/Contents/bin/cmake-gui” –install=/path/to/bin

 

I opted for the second option, mostly because it was simplest and I want cmake to available everywhere and every time I need it

sudo “/Applications/CMake.app/Contents/bin/cmake-gui” –install

 

Where are your gcc, gfortran, and clang?

 

Once we have all these and before you start, we have to check few things. In this installation, I wanted to use gcc, gfortran, and clang for my setup, so get to know where they are located in your system by using the ‘which’ command

The Installations

 

In the terminal, go to the UQTk_v3.0 directory

  • mkdir builddir
  • cd builddir/
  • cmake -DCMAKE_Fortran_COMPILER=/usr/local/bin/gfortran -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_CXX_COMPILER=/usr/bin/clang ../../UQTk_v3.0

 

Replace green text with your own path.

Once the cmake configuration is complete successfully, type make

All will go ok, until you hit this error.

In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:23:

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/

MacOSX10.11.sdk/usr/include/unistd.h:510:14:error:declaration of ‘optarg’ has a different language linkage

extern char *optarg; /*

getopt(3) external variables */

^

/Users/sukhbindersingh/PROJECTS/UQTk_v3.0/cpp/lib/include/gen_defs.h:49:14: note:

previous declaration is here

extern char *optarg;

^

1 error generated.

make[2]: *** [cpp/lib/CMakeFiles/uqtk.dir/tools/multiindex.cpp.o] Error 1

 

Well this is easy to solve and I think it’s an error in the file. Open gen_defs.h file from cpp/lib/include and comment the following line as shown.

extern char *optarg

 

 

Another error that I faced was the missing gfortran library because of no LIBRARY_PATH.

ld: library not found for -lgfortran

clang: error: linker command failed with exit code 1 (use -v to see invocation)

make[2]: *** [cpp/app/gen_mi/gen_mi] Error 1

make[1]: *** [cpp/app/gen_mi/CMakeFiles/

gen_mi.dir/all] Error 2

make: *** [all] Error 2

 

This meant my libgfortran.a was not in the LIBRARY_PATH, so let’s find where the library is and then set the LIBRARY_PATH environment variable pointing it to that folder.

Locate libgfortran*.a

sudo find /usr -iname ‘libgfortran*.a’

 

Found them here.

/usr/local/gfortran/lib/i386/libgfortran.a

/usr/local/gfortran/lib/libgfortran.a

Set the environment variable to that folder.

export LIBRARY_PATH=/usr/local/gfortran/lib/

 

Compilation should continue smoothly and at last type make install and you should have everything installed.


Type ctest to check your installation


Recap

 

So here’s recap of the steps.

  1. mkdir builddir
  2. cd builddir
  3. cmake -DCMAKE_Fortran_COMPILER=/usr/local/bin/gfortran -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_CXX_COMPILER=/usr/bin/clang ../../UQTk_v3.0 [Without python]

    or cmake -DPyUQTk=ON -P-DCMAKE_Fortran_COMPILER=/usr/local/bin/gfortran -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_CXX_COMPILER=/usr/bin/clang ../../UQTk_v3.0 [With python]

  4. make
  5. make install

I faced two problems,

  1. error:declaration of ‘optarg’ has a different language linkage Solution: comment the offending line.
  2. ld: library not found for –lgfortran clang: error: linker command failed with exit code 1 Solution: export LIBRARY_PATH=/usr/local/gfortran/lib/

Using PyUQTk option gave me lot of problem, have delayed installing python binding for now due to lack of time.

Well now this is done, will spend couple of weekends looking at the library, you can fork the entire library from my github page. It has all the libs, so if the compliers are similar, they can be used as is without going through all this steps.

Get Mount Drive from mount path in windows

Things that are simple in Linux and UNIX based system always end up becoming a head ache in windows, but unfortunately most of our work-life runs on windows.

And if you don’t have admin rights to install specialized modules in your python installation, then here’s a hacked version of routine that should do the job.


def get_mount_drive(path):
    os.system(r'net use > t.txt')
    lines=open('t.txt').readlines()
    for line in lines:
    if path in line:
       return lines.split()[1]
    return None

Mixing debug and release libraries in Visual Studio

buffer overrun

Mixing debug and release code is bad practice.

Different compiled versions can depend on different fundamental parts of the C++ runtime library, such as how memory is allocated, structures for things like iterators might be different, extra code could be generated to perform operations. In particular, the std containers and iterators are different and incompatible and do not let this work.

Sometimes one get the following error

A buffer overrun has occurred in Program_64.exe which has corrupted the program’s internal state. Press Break to debug the program or Continue to terminate the program.

For more details please see Help topic ‘How to debug Buffer Overrun Issues’.

So are we stuck?

But what if you have inherited the release library and do not have the source code. You are stuck for development and if the compiler is not letting you to use these libraries in your debug development code.

But we are not entirely stuck, here’s a workaround that worked for me.


// the offending libraries
#include "quad.h"
#include "uqtktools.h"

#include "mytypedef.h"

using namespace std;

The work around

#ifdef _DEBUG
#define DEBUG_WAS_DEFINED
#undef _DEBUG
#endif
// the offending libraries
#include "quad.h"
#include "uqtktools.h"

#ifdef DEBUG_WAS_DEFINED
#define _DEBUG
#endif

#include "mytypedef.h"