How to resolve this pandas ValueError: arrays must all be same length - Resolving "arrays must all be same length " pandas error is easy. Here's how
Put an Image Behind your matplotlib plots - Here’s a quick one. Problem. You want to add pretty graphics in the back of your data. How to do this with matplotlib? Solution Simple. Here’s the result
Some Useful pytest Command-line Options - I love pytest. Pytest is a testing framework which allows us to write test codes using functional python and functional python is awesome. Why use PyTest? There are many reasons to use pytest here are some that I feel are important. Very easy to start with because of its simple and easy syntax. Less Boilerplate […]
Python Logger Printing Multiple Times - This is my standard boilerplate code for adding any logging functionality in an app. This makes starting and working on projects super easy. But sometimes if the project involves multiples modules, there is an annoying little thing that the log is printed multiple times on the console. In one particular project, which was using multiple […]
Standard setup.py - Yes yes I know we should use poetry and other packaging mechanisms, but for simple small projects at home or for a simple application the setup.py is a good place to begin Here’s a sample setup.py that I have used in many of my personal projects Standard Setup.py This is useful and is always a […]
JSON to Named Tuple - You have a JSON file and you are tired of getting the JSON just as a plain vanilla dictionary, then the following code using the namedtuple available in the collections module in standard python can come to your rescue Here’s an example Observe the convert definition. Now one can access its elements like Hope this […]
Rendering Matplotlib Graphs in Django - If you have seen my post on the expense Django app, the dashboard is rendered using matplotlib’s static files which are passed on runtime to the webpage HTML. See this here in action. All the png images are generated on the fly and sent to the browser to display. Here’s the simple code stript down […]
Error: invalid command ‘bdist_wheel’ - This is a common problem that I have seen while building a wheel in python. Mostly this is the case because wheel is not a standard python library distributed with python. Problem Solution That’s it. Install wheel from pypi. Pair this with these posts.
Parsing Date Like a Boss with Python - Daily planning is one thing that I do regularly. For office related work, most of this is done in outlook and doing all this with GUI would be too distracting, so I have this cmd utility created in python. Here’s how this works Type on cmd and it creates the appropriate meeting schedule. Here’s a […]
Python is Beautiful - Python is Beautiful. Give it your love.
Heart Equation - If you are following this blog for a long, you will know, there is always one heart post every other year mostly around valentines day, The posts all almost always based on this equation. Maths is so beautiful. Well here’s this year’s rendition of the heart using the previously discussed comet plot. The code for […]
Let It Snow- the python code. - Create this beautiful and calming snowfall animation with python.
How Not to Miss any Outlook Appointment -Introducing Email Butler - I work on two PC’s. One is my local machine and another is a virtual remote desktop (VDI) machine where I connect remotely. The remote machine is where I spend most of my workday but the local machine has the primary email where all office-related meetings and other communications come by. So have to manage […]
Let It Snow – Revisited - As we enter the holiday season and into the new year, wanted to revisit an old post that I did back in 2011 with the same title but the animation was done in FORTRAN. Here’s a similar implementation in python and matplotlib. In Fortran I needed pgplot for the animation, in python, only standard packages […]
Treemap in Python - Tree Maps are primarily used to display data that is grouped and nested in a hierarchical. A treemap is one method to simultaneously display the magnitude of the major categories – as well as the magnitude of the larger subcategories in one visualization. Ben Shneiderman introduced treemaps in the early 90s as a visualization metaphor […]
Definition in Command Line - Kids are much smarter these days. Working from home I had this experience. As soon I settle down to work my younger one will come to me and ask “Daddy what does ‘fluff’ mean? “ Sometimes I answered but most of the time, being busy with my own new work-from-home setup, the default answer was […]
How to check if a process with a given PID exists with Python? - While working on building a scheduling system, this was a common requirement. Fortunately, python offers a few ways to accomplish this. Here’s are three ways to find out if the process is active with python. One uses an awesome 3rd party library psutil and the remaining two just use the python standard library. Solution 1 […]
Happy Diwali - A few years back wrote this post during Diwali on how to use animation in matplotlib. The result was this simple image. This year just as a throwback created this new gif. Happy Diwali. Here’s the code if you are interested in learning how it’s done.
How To Make Your Computer `Chime` - When I was in year 4, my dad got me a Casio digital watch. Its hourly chime and alarm were my favourite features. Those days watches were the smart techs we had. Three years back got myself a retro Casio digital watch just for the chime feature again. Since working from home, the watch has […]
Calendar in Cmd - In my quest to make my windows system mostly behave like Mac, i keep adding things that i miss in windows like the ability to see the calendar in terminal. In macbook pro, typing cal anywhere in the terminal gives you the calendar. Python comes to my rescue in all these quests to turn windows […]
One cool use of functools.partial - The functools module is for higher-order functions: functions that act on or return other functions. In general, any callable object can be treated as a function for the purposes of this module. Instead of using lot of os.path.join This pattern is much more useful and more so if you want to use the path multiple times. Discovered […]
Comet Plot in Python - In the summer of 2016, while working on a project with Matlab, I discovered the cool plotting function in Matlab, comet. You have to see the plot to experience this. Ever since wanted to recreate that plot in python, so here we go. Here’s a code to display the above plot Here’s the code for […]
Number Puzzle and Few learnings - In April 2013, in my quest for sharpening my python skills, I wrote the below program to replicate the numbers puzzle game I have played during my childhood. Found this on my old hard disk which I was cleaning recently. Looking at it now I find lot of issues with it. Use of global variables […]
Simple Gems of Python - This functionality of Python is one I have applied in many of my pet projects and scripts. Do you know about it.
Two Important Reason for Testing are … - Back in 2020 during the start of the first wave in the UK, before the lockdown, I gave a quick talk on testing with Pytest to a few developer’s colleagues in the office. Few slides were titled why test and elaborated on various (more like 8) reasons for why we should write tests? Two reasons […]
Best Practice for Writing Tests for Argparse - If you write serious python apps for the end consumers, you ultimately end up writing argparse or similar CLI tools. Once written, this needs to be unit tested, so what is the best way to test the parsers? There are many different ways that can be done based on one’s application. For me, I have […]
How to Get System Ideal Time for Windows System - Two years ago faced a problem where my python application needed to know if the system is ideal or not. After experimenting with several options, finally settled on the following solution via this All done with python standard library. I love python’s battery included philosophy. Here’s a quick demo of the function I have used […]
Intel MKL FATAL ERROR - Last August, I got a new system from the office and I took this opportunity to upgrade my python 3.5 to python 3.7. Everything was working fine except for this error I searched for mkl_intel_thread.dll and could see them in the correct env folder. Many in StackOverflow advised moving the dll one folder up. Since […]
4-7-8 - This was one of the forwards from the HR team when the pandemic started for everyone’s well being. 4-7-8 Breathing technique to calm your mind. It was cool and just for the sake of it, did develop a mini-app in python that helps do this from command line. Here’s a demo. Works in windows, Linux […]
Deploying Django App with PythonAnywhere - Recently wanted to deploy a learning app I have developed for my kids on the web. The main reason behind this was to make the app accessible from anywhere and from any device. This was necessitated by the our travel back to kids grandparents, where the laptop was not always online and on. After research […]
How to Determine Memory usage of a Python Function - In enginnering software development or other software developement projects, most programmers are solving a problem and this always starts by writing a naive implementation and then optimizing it in various ways. Many a times, instead of focusing on timing, we are also interested in memory usage so we can explore those tradeoffs of caching a […]
Determine if a given string is a URL - Recently while working on a side project in python and web, needed to find if a string is a url or not? Here’s a solution using python standard library Love this simple small snippets using the standard library. Do you know of such a hidden gem?
Managing Housing Society Expenses - Last summer, while in home quarantine, received an update from the housing society committee on the general group, that nominations are invited for new managing committee. I ignored it. Come September, learnt that many ignored the same and now by way of a lottery system I was one of the chosen few to take up […]
Reading/Writing wav files with Python - Problem: You want to read and manipulate sound (*.wav) files but since you are on a company-controlled system, you are not allowed to install pyaudio or other excellent audio packages. How do you solve this using basic standard python package? Solution: Example: These two functions recently helped me in a personal machine learning project. I […]
Intersect - Last month, someone emailed me after checking out my post on intersection of two curves in numpy. He wanted to use the code but enquired if the code was available on the pypi for easy installation. This enquiry prompted me to look at the code that I wrote 4 years ago. To my surprise, this […]
Starter code for Using Tensorboard - I have been using keras for multiple years now at home on my personal projects, one things that I have used very little is the tensorboard, mostly because I run the models on my home system and the console output more than satisfies my needs. But now I am planning to use it regularly. So […]
Few ipython %run magics I can’t live without - If you are not using ipython I strongly encourage you to stop reading and get ipython. From the time I have discovered it, 8 years back, i haven’t looked back. This is my default python console. Even jupyter hasn’t replaced it. Why? you ask. And here’s a list of ipython’s %run magics options that I […]
Working with PDF in python - I have a love hate relationship with the pdf format. I like the convenience of the portability but its hard to work with you if you need the data. And because of the portability, most of data coming my ways mostly is in pdf format, so i have come to rely on the an excellent […]
Record Screen With Python - There a tons of screen recording software available in the market, but you don’t want to install one more software for just one usage. Or sometimes installing a freeware or a paid ware is not allowed. What to do? you improvise. If you know and have python on your system, you are in luck. This […]
Winsay - The thing I like about MacBook is that it comes with simple features/apps in-built that makes life easy. And being forced to use windows at office I have tried to use python as the bridge to fix these gaps. One of the feature I like about mac for entertaining kids, and to relax my eyes […]
Using genfromtxt instead of pandas - When reading a csv file, pandas have become my default, but most of the time, this adds pandas dependency just for reading CSV, so I tend to use numpy's genfromtxt. This is a versatile and fast option to use. To read mixed-type data, the dtype=None is a good option to use. Using dtype=None is a […]
Numpy to Json - I don’t know why I have this snippet of code as a text file. This is something that I have used in few of the api endpoints. A pattern so common, so putting it here as a reference.
Few Not So Common but Useful git commands - Git is a versatile tool. It has a lot of things that can make it productive to use. Here’s a list of a few commands that I use in git on a monthly basis for my git repositories. git fsck -> Files system check git gc -> Garbage collection git diff –stat -> Only stats […]
Spellings: How to get good at it? - As my school friends will attest, spellings were never a strong suit for me. Most of this could be because I was lazy and never paid attention to it when reading. I liked reading but never looked at the spellings. I forgot about this until few years back when I began finding my kids getting […]
AVI to GIF with python - Problem: You have an AVI file that you want to convert to gif. Lets complicate this question by constraining that you want to get some frames and you want to crop a certain area of the video. Solution using python Here’s the simple code to do the conversion of the entire video to gif Here’s […]
Determining screen locked of a system using python’s standard library - I think, apart from the ease of use, python’s batteries include philosopy is one of the reason its has become so popular. Here’s another cool functionality that we needed in one of our app that was trying to maximise the usage of computing resources when the user has locked his computer. The problem, Get to […]
File and Folder Comparison with Python - Python standard library modules are incedible. There’s a small gem to compare files and directories. Its useful Say you have two ascii files and you want to do a file comparision, don’t worry, use python. To compare two directories, use filecmp module has utilities for comparing files and directories. It consists of the following. Refer […]
One-liner in Python to get the Directories - Python’s standard library is powerful and can be used for a lot of good things. Here’s a simple one-liner to collect all the directories from a path. What are your favourite one-liners that you tend to regularly use in python?
Get All Info About a Python Environment - Conda makes creating environments easy and if you are anything like me, over the course of time you end up having many enviroements and it becomes difficult to knwo what is what. Basic hygine is to make environemnet names unique and descriptive. But even then knowing what an environment has becomes difficult. Here’s a general […]
Starter Argparse and Setup.py templates - I tend to develop my python scripts as small apps, this way I can call and use them from every where in command line without relying on the calling the python scripts Here’s a template of the setup.py and argparse template that I always start with. Setup.py example is for making the application as a […]
Few common things to look out for while converting python 2 to python 3 - If you are converting your python2 code to python 3 manually. apart from the print statement, here are few things that i encountered. xrange is not available in python 3, simply used range 2. dict_keys in python 2 was a list and were iterable, but in python 3 you get the following error If you […]
Split text after nth occurrence of character - python never fails to amaze me. You have to keep using and and you find these little gems hidden in it. Suppose you have a string like this you want to split the numbers and the tags, meaning at the fourth occurance of comma, how will you do it? simple. this returns a list like […]
Get Outlook Entries With Python 3 - This was an old post, and was written in python 2, but that refused to work in python3 as pointed by win Today found some time to look at this and fixed the code. So here’s new improved code that works both in python 2 and python3. The new code gives user the ability to […]
Memory Profile Your code with Ipython - Lets say you have a function that you want check the memory usage in python. This can be evaluated with another IPython extension, the memory_profiler. The memory profiler extension contains two useful magic functions: the %memit magic and the %mprun function. %memit magic gives the peak and total memory used by a function, while %mprun […]
Get Activation from scikit-learn’s neural network model - I have a simple multilayer perceptron that I use on my work computer. It works well with the and has 95% accuracy on top 5 basis. It’s a delight to see it work, but I want to get more insights on what is happening inside it, one way to make it work is to see […]
Shutil make_archive to Rescue - Shutil to rescue I am amazed at the versatility of the shutil library, one usage that i discovered recently was its ability to create archives. Previously I was always using zipfile module from python but make_archive function of shutil is such an intuitive function to use. With a single line you can take backup of […]
Computer Name in Python 3 ways - Problem: Get the computer name in python without using any external module or library Solution:
Shutil to Rescue - As part of my work, need to run simulations which are driven by python’s sub process module is the work horse for this. But there’s a problem. On Windows,the subprocess module doesn’t look in the PATH unless you pass shell=True. However, shell=True can be a security risk if you’re passing arguments that may come from […]
Get Screen Size With Python Standard Library - Batteries included!! The best thing about python is the batteries included. ctypes is one such versatile module. Here’s how to use ctypes to get screen size
Sound Alarm When Program Execution Completes in Python - Often one faces a situation where your code takes extremely long to run and you don’t want to be staring at it all the time but want to know when it is done. In engineering analysis, simulation take a long time to run and the python driver program take a long time to finish and […]
Simple numpy trick to check if vector is increasing - Problem: How to check if a 1d vector is increasing……. Solution: where xp is a vector of numbers. I love numpy. When is it going to a be standard module in python?
Shelve it with python - One of the little gems hidden in python standard library is shelve. The shelve module can be used as a simple persistent storage option for Python objects when a relational database is overkill. The shelf is accessed by keys, just as with a dictionary. The values are pickled and written to a database created and […]
Participate in the 2018 Python Developer Survey - If you use python in your work or hobby projects, please consider participating in the 2018 python developer survey. Reposting a PSF-Community email as a PSA Excerpt from an email to the firstname.lastname@example.org and email@example.com mailing lists: As some of you may have seen, the 2018 Python Developer Survey is available. If you haven’t taken […]
Outlook Calendar Entries for a Day with Python - For me it is not the mobile. It is the email, which is the most distraction application in office. The variable reward system has become a habit. I tried closing it but then I keep missing meetings etc. So had to keep it open just so the reminders for meetings show up. Only reason I […]
Mosaic Plot in Python - I am sure everyone one of us has seen charts like this. Any management training you attend, a version of this chart is bound to sneak up in the presentation, often in lecture notes or hands on activity. These charts are a good representation of categorical entries. A mosaic plot allows visualizing multivariate categorical data […]
Timeline in Python with Matplotlib - Was documenting a series of events and after the list was complete, thought of showing them on a timeline. After an hour two, fiddling with python standard libraries this is what I had. Format of the data The code as always available at this GitHub
Detect Outliers with Matplotlib - Problem: You have a huge large multivariate data and want to get list of outliers? Outlier detection is a significant statistical process and lot of theory under pining but there is a simple, quick way to do this is using the Inter-quartile (IQR) rule. Read the linked PDF for a simple example summary Solution: bloxplot_stats […]
Q in Statsmodels - If you are coming from R, you will love the formula API of statsmodels that work in a similar way. I love it and been using it for quite some time since last year. It’s good to test quick regression and GLM models and works on pandas dataframe which, at least in my case, are […]
Numpy Arrays to Numpy Arrays Records - Recently working with hdf5 format and the datasets in the format requires the arrays to be records. What does that mean? NumPy provides powerful capabilities to create arrays of structured datatype. These arrays permit one to manipulate the data by named fields. One defines a structured array through the dtype object. Creating them from list […]
Cross Validation Score with Statsmodels - Recently was working with a limited set of data. Using statsmodels , employed a regression model on the data. To test the confidence in the model needed to do cross validation. The solution that immediately sprang to mind was the cross_val_score function from sci-kit learn library. However, that function is not applicable on statsmodels object. […]
Parsing date – a simple example - Problem: Let’s say we have a data like the below image, it can be anything, from IOT sensors data to something like the below expense log. If the data has day, month and year as separate columns, as shown below, how to read this kind of data log with pandas and create date as the […]
Variable length list to Numpy array - Suppose you have a variable length list and you want to convert it to a numb array What is the efficient way to convert this list to a numpy array? My first answer was using pandas and this is what I did? This worked and I moved on to my other problem, but then realised […]
Panda Time index vs DateTime and Inconveniences - Sometimes convenience becomes a handicap. Saw this first hand later last month. Was so used to using Pandas DataFrame and the Timeindex object that when I had to move back to a system which didn’t have pandas I was struggling to get a simplified day of year, day of week and week of year from […]
Updated Binary STL File Reader - A recent comment on the post titled binary stl file reader in numpy prompted me to revisit the code and while I was looking at it , I updated the same for code to view the loaded stl file. The above picture shows the result. The code as usual available on github from here
Joint plot with Matplotlib - Today I am releasing a simple module to create joint plot with Matplotlib on github. Joint plot is available in the excellent seaborn library but unfortunately it’s not always available on many systems. Recently I needed this functionality, so wrote this simple module with matplotlib. The functionality is almost similar to seaborn but with limited feature. […]
Quarterly Results Analysis with Python - Every quarter when results come out, I spend sufficient hours looking at the financial results of the stocks I am tracking to elicit nagging comments from my better half. So developed this simple python script which does the analysis and generates me the PDF which I email to myself to look at during office commute. […]
OD Command in Windows with Python - I have seen OD command for last two years but had never bothered to look it up. It always remained under the surface. Life went on well without it, until now. During a recent debugging crisis, had to resort to this command for an efficient way to check binary results. All the while had to […]
Get Mount Drive from mount path in windows - Things that are simple in Linux and UNIX based system always end up becoming a head ache in windows, but unfortunately most of our work-life runs on windows. And if you don’t have admin rights to install specialized modules in your python installation, then here’s a hacked version of routine that should do the job.
Few Python Regex for FORTRAN Code - Most of my day job involves Fortran code and finding quick details is a regular need. Here are four regex in python that I have been using for some time now. Sharing it here. To Find Subroutines and Functions To Find All Call Statements To Find Variables (Integer, Double Precision, Logical, Real) […]
Simple trick to read mixed format data with numpy’s genfromtxt - Suppose your data is like the following Let’s load it with numpy using genfromtxt This prints the following Numeric data is read but not quite as what we wanted. Notice the NaN. You can supply genfromtxt the datatypes using the keyword format like dtype=([(‘f0’, ‘<i4’), (‘f1’, ‘<i4’), (‘f2’, ‘<i4’), (‘f3’, ‘<f8’), (‘f4’, ‘|S14’)]) But that […]
Quick Gantt Chart with Matplotlib - The problem: You need a quick Gantt chart for a quick proposal report and you dint have any project planner software installed. Solution: While there are many different ways, you only have access to python, well Here’s a simple Gantt chart plotter with just matplotlib. Not for rigorous use but a good substitute to make […]
Biplot in Python revisited. - Mark sent a recent email complaining the previous biplot code not working. Though I was not able to replicate his errors,but from the error message figured the error was due to the PCA numbers supplied. That reminded me of the simplification task that I intended to do on the previous version to make it work […]
Visualizing 3d Triangles With Pure Matplotlib Function - The problem: STL file is read, you want to do a quick sanity check if it’s correct and don’t have access to VTK or third party STL viewer. Solution: Use matplotlib to visualize the STL. Here’s a simple matplotlib script to visualize the STL triangles.
Matlab to Python – some code examples - Two years back, I was converting a matlab script to python, here are some of the errors that I encountered during the conversion. Found them documented in that converted script, posting them here for wider audience. Matlab to Python 1. () to  SyntaxError: can’t assign to function call 2. 1 to 0 IndexError: index […]
Pdf With Matplotlib - I thought everyone knew about but i was surprised this this little feature of matplotlib is not that known as widely as I assumed. We all know we can save a plot from matplotlib to pdf but there other little feature hiding in the backends where we can write out a multiple page pdf Here’s […]
Biplot with Python - I have plotted Biplot in Matlab and have created it using fortran in the past. Last month, while playing with PCA, needed to plot biplots in python. Unlike MATLAB, there is no straight forward implementation of biplot in python, so wrote a simple python function to plot it given score and coefficients from a principal […]
FEM in Python A Simple Start Guide - Wrote this a couple of months back. Yet another tutorial in python, if you are interested in finite element analysis. Nicely goes with this excellent tutorial on FEM. Click the below link to view the tutorial. FEM with Python
What can Software Teams Learn From Aviation? - This pycon talk combines my two passions, aviation and software development. Grab a cold coffee and watch this very interesting talk. Just a little over half an hour, if you have any interest in any of this two stream, you will like the talk. What can Python-based software teams learn from aviation? Why should software […]
Exploring Matplotlib Styles - Last week got some free time. Used it to upgrade my python installation on Mac, a long awaited task. Looking at the upgrade log, was most excited to finally see the new version of matplotlib. So launched it and went straight to the new style package. Matplotlib is great at graphs but the default style […]
A Gentle Introduction to the Finite Element Method - Almost every engineer today have some knowledge of theoretical and practical aspects of FEM. And many have played with various software packages at some point of time. Well that is enough if you just want to perform analysis but if you want to truly understand the mathematical aspects of FEM and want to get a […]
Quick Intro to 2D and 3D Visualization in Python - Posted a quick tutorial introducing visualization in python. This tutorial offers a quick look at some common 2d and 3d Visualization available in python. This tutorial is meant to give anyone a quick start in matplotlib and mayavi.mlab. The tutorial was written as part of CSRP program of AeSIAA which I am volunteering as a […]
Quick Basic Python Tutorial - I have just posted a Quick Basic Python Tutorial on github. View it here Quick Introduction to Basic Python. Short and simple for those who cannot wait to get started.
Displaying Truss - The truss program that was posted two weeks back, looked incomplete without a proper visualisations, so here’s a small VTK function written to display the truss with any quantity like stress/reactions or displacement.
Python Over Other Languages - Found this interesting Infographics on Python over other languages. There is no mention of fortran, but everything said for c++ applies to fortran via Intellipaat
Analysing Trusses – a python program - Was digging into my laptop and found this Truss program written in python. Computes, displacement, stresses and reactions.
Simple Python code to Extract Fortran Routines and its Argument - Suppose you got a long fortran file full of code, and want to parse it and get the subroutine/functions defined in the file along with its argument. Here’s a simple python code to extract all this info in one go
A Simple but useful Python tip. - Want to convert a numpy arrays containing coords from x1,y1,z1 x2,y2,z2 . . . xn,yn,zn to x1 y1 z1 x2 y2 z2 …. xn yn zn Example
An Example of Model View Controller Design Pattern with Tkinter Python - Model-view-controller (MVC) is the design pattern for successfully and efficiently relating the user interface to underlying data models. This is a useful pattern for the reuse of object code and a pattern that allows to significantly reduce the time it takes to develop applications with user interfaces. I have used MVC pattern in my ICP […]
Class version of Spring FEM in Python - Was cleaning the laptop and found this class version of the spring FEM code that I have posted earlier. Not the best implementation but Posting it here just in case…
Fortran from python – Python Binding for Fortran Code - As an aeronautical engineer, I received fortran as a legacy. From NAL to working for Rolls-Royce, work life revolves around fortran. This is the language that I have been programming for 12+ years now. That explains this many fortran related posts here and here. But from couple of years, I am falling in love with Python. Here […]