When in Doubt, Subtract

One of the most useful idea from Nassim Nicholas Taleb is

Via Negativa: Removing is superior to Adding

In concrete terms.

– Less can be More

– Don’t add, subtract

– Eliminate the Artificial

– Eliminate the Unnatural

– Eliminate the Unnecessary

When in doubt, SUBTRACT.

This can be appied to everything in life. Do you agree?

Grep in windows

Mac and Linux have great grep command to search within files, but in windows, this is difficult.

I use the following, not well advertised, command to do things that grep does.

findstr /s /m /i /c:”reference condition” *

Searches for strings in files.

FINDSTR [/B] [/E] [/L] [/R] [/S] [/I] [/X] [/V] [/N] [/M] [/O] [/P] [/F:file]
        [/C:string] [/G:file] [/D:dir list] [/A:color attributes] [/OFF[LINE]]
        strings [[drive:][path]filename[ ...]]

  /B         Matches pattern if at the beginning of a line.
  /E         Matches pattern if at the end of a line.
  /L         Uses search strings literally.
  /R         Uses search strings as regular expressions.
  /S         Searches for matching files in the current directory and all
  /I         Specifies that the search is not to be case-sensitive.
  /X         Prints lines that match exactly.
  /V         Prints only lines that do not contain a match.
  /N         Prints the line number before each line that matches.
  /M         Prints only the filename if a file contains a match.
  /O         Prints character offset before each matching line.
  /P         Skip files with non-printable characters.
  /OFF[LINE] Do not skip files with offline attribute set.
  /A:attr    Specifies color attribute with two hex digits. See "color /?"
  /F:file    Reads file list from the specified file(/ stands for console).
  /C:string  Uses specified string as a literal search string.
  /G:file    Gets search strings from the specified file(/ stands for console).
  /D:dir     Search a semicolon delimited list of directories
  strings    Text to be searched for.
             Specifies a file or files to search.

Thankfully windows 10 has bash command’line, I haven’t used it but hope all this will be unnecessary in  it.

Do you have a commandline trick that you always come back to?

Yes or No

Photo by Pixabay on Pexels.com

When you are off a project and put into a pool of resources for anyone to pick, every third email is an offer to say yes or no to something. Sometime a new technology, sometimes a new customer or sometime exciting responsibilities.

It’s always is a question of saying yes or no. This situation reminded me of a quote I read from James Clear.

When you say no, you are only saying no to one option. When you say yes, you are saying no to every other option.

No is a choice. Yes is a responsibility.

Dole out copious amount of No. Be very careful with Yes.

The choice for me was easy as I have already said yes to something else!

Few Things to Know in Unittest

I have completely shifted to pytest now, but it was standard unittest module that I started, like many others.

Pytest is simple, intuitive and easy to work with but it always have to be installed as an additional package, sometime installing new things is not possible. (like on rigid office installations :()

So for that unittest, the standard package is a good option..

Apart from the usual unittests things, I have found the followings useful for real world unit testing of apps.

Putting them here as a reference

    def test_gauss_lobatto_quadrature(self):

    def test_mc_orthogonality(self):

The following decorators and exception implement test skipping and expected failures:

Unconditionally skip the decorated test. reason should describe why the test is being skipped.

@unittest.skipIf(condition, reason)
Skip the decorated test if condition is true.

@unittest.skipUnless(condition, reason)¶
Skip the decorated test unless condition is true.

Mark the test as an expected failure. If the test fails it will be considered a success. If the test passes, it will be considered a failure.

What other things will you add to this list?

Sir I don’t sell to fish.

I don’t remember where I have read this small anecdote years ago, but got reminded of this last week while interacting with someone.

There was a salesman selling fishing tackle. Looking at the variety of colours, one person asked “oh my god, some are purple, some pink, do the fish really like them?”

He replied. “Sir I don’t know. I don’t sell to fish?”

A good story to remember for everyone, as we all are selling something every time.

Determining screen locked of a system using python’s standard library

I think, apart from the ease of use, python’s batteries include philosopy is one of the reason its has become so popular.

Here’s another cool functionality that we needed in one of our app that was trying to maximise the usage of computing resources when the user has locked his computer.

The problem,

Get to know if the screen is locked

def screen_locked():
    Find if the user has locked their screen.
    user32 = ctypes.windll.User32
    OpenDesktop = user32.OpenDesktopA
    SwitchDesktop = user32.SwitchDesktop

    hDesktop = OpenDesktop("default", 0, False, DESKTOP_SWITCHDESKTOP)
    result = SwitchDesktop(hDesktop)
    if result:
        return False
        return True

Function Vs Form

Photo by Daria Shevtsova on Pexels.com

Most of last week was spent packing. Moving suitcases around the house, weighing, organizing stuff in them and fiddling with locks.

While fiddling with a forgotten number lock on a suitcase, got reminded of an anecdote I read a while back about suitcases getting the wheels. 

Suitcases with wheels are ubiquitous  now but they were not so common a couple of decades ago.

In ancient Rome, soldiers used leather messenger bags and satchels to carry food while riding across the countryside. At the same time, the Romans had many vehicles with wheels like chariots, carriages, and wagons. And yet, for thousands of years, nobody thought to combine the bag and the wheel. The first rolling suitcase wasn’t invented until 1970 when Bernard Sadow was hauling his heavy luggage through an airport and saw a worker rolling a heavy machine on a wheeled skid.

There was plenty of innovation going on. Throughout the 1800s and 1900s, leather bags were specialized for particular uses—backpacks for school, rucksacks for hiking, suitcases for travel. Zippers were added to bags in 1938. Nylon backpacks were first sold in 1967.

Despite these improvements, the form of the bag remained largely the same. Entrepreneurs were so locked in on what a suitcase should look like that they didn’t consider what a suitcase was meant to do. Innovators spent all of their time making slight iterations on the same theme.

What looks like innovation is often an iteration of previous forms rather than an improvement of the core function. While everyone else was focused on how to build a better bag (form), Sadow considered how to store and move things more efficiently (function). When optimizing for function instead of form, the addition of wheels became obvious.

via jamesclear.com

What are things around you which are optimised for form and can use optimise on function?

Bloody windows: Create conda environments

Bloody windows

One of my collegue at office really really hates windows, every time he is forced to use windiwos, I have heard him mutter “bloody windows”

Sometime back I had the same desire.

This is the batch script, which works perfectly fine if I use the command in the cmd window but it failes in batch mode.

Old batch file

rem Create conda environment
echo Creating conda environment
conda deactivate
conda env create --file environment.yml --force --quiet

After a lot of head scratching, trying to figure out what is happening, found that I have to use call.

New batch file

rem Create conda environment
echo Creating conda environment
call conda deactivate
call conda env create --file environment.yml --force --quiet

Turns out call is needed.

call activate %name%

I’m assuming that activate is a batch file. If you call it, processing will return after that batch is finished. Without the call, execution is transferred to activate and ends when activate ends

Things That You Didn’t Do

A quote I was thinking for the last week..

Mark Twain, the famous American author, is quoted as saying –

Twenty years from now you will be more disappointed by the things that you didn’t do than by the ones you did do. So throw off the bowlines. Sail away from the safe harbor. Catch the trade winds in your sails. Explore. Dream. Discover.

Solving UnicodeDecodeError while reading csv in python

Recently tried reading a csv file with pandas and got this error

fname = r"C:\Users\singh\Desktop\prediction.csv"
data  = pd.read_csv(fname)

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 7: invalid start byte

After much head scrating, reading various stanckoverflow post and the reading the pandas docs, found the solution .

data = pd.read_csv(fname, encoding = 'unicode_escape')

yes just a single paramenter encoding = ‘unicode_escape

Posting it here in the hope it helps others.

A Proper Bhangra Gif

While in lock-down due to COVID-19, our standup meeting morphed into daily cafe with one social things that we did as a team. A few weeks back the theme of the social cafe was music you like.

For this I was going to present my favorite kind of music, which is bhangra style music. For this presentation on my music theme, I needed a good animation of Bhangra. After few google searches, all I got was cartoonist versions of the dance form.

All of them looked childish and cartoonish. I couldn’t find a decent one that was actually depicting the dance form in its true essence, but that search led me to this YouTube video.

I liked the simple background. As I did not have access to a screen capturing software on my work computer, launched python and created this gif using it.

Code and steps I took to convert this you tube video to the gif that I wanted if something for a different post.

So here’s a proper Bhangra Gif, Thanks to the youtube video for all the credits.

Be sure watch the Youtube video that has the music with this dance.

Spiral Approach to Learning

Photo by Karolina Grabowska on Pexels.com

Being a parent, I am always on the lookout for ideas/ stuff that can help kids learn.

In our house we already have a spaced revision program [written in python] that the kids now enjoy practicing their spellings on. Here’s an video of one such session

Obviously you can’t learn (or teach) everything at once … but you have to start somewhere. This leads to two important questions: (1) where to start, and (2) when to move on to the next topic.

To answer such questions, the only thing that has ever made sense to me is the following:

Spiral approach: Learn a few rough ideas, then spiral back, making connections between the ideas. Then keep spiraling back again and again, gradually adding new ideas, refining the ideas, and making more connections, checking to see in what ways each idea is consistent (or inconsistent!) with the others.

Therefore the answer to question (1) is that the starting-point doesn’t matter very much. After a few trips around the spiral, nobody knows or cares where it started.

Question (2) is super-important, because it will come up again and again, every day for the rest of your life. Each time, the spiral approach provides a partial (but not complete) answer. It says that you do not need to master one topic before moving on to the next. Plan on coming back.

via The Spiral Approach to Thinking and Learning

I agree with this philosophy of learning and do most of my learning this way. What is you take on this?

File and Folder Comparison with Python

Python standard library modules are incedible. There’s a small gem to compare files and directories.

Its useful

Say you have two ascii files and you want to do a file comparision, don’t worry, use python.

import filecmp

# Check two files
assert filecmp.cmp(base_reduced_bdd, bdd_file, shallow=False) is True

To compare two directories, use

x = filecmp.dircmp(dir1, dir2)

# prints a report on the differences between dir1 and dir2

filecmp module has utilities for comparing files and directories.

It consists of the following.


    cmp(f1, f2, shallow=True) -> int
    cmpfiles(a, b, common) -> ([], [], [])

Signature: filecmp.cmp(f1, f2, shallow=True)
Compare two files.


f1 -- First file name

f2 -- Second file name

shallow -- Just check stat signature (do not read the files).
           defaults to True.

Return value:

True if the files are the same, False otherwise.

This function uses a cache for past comparisons and the results,
with cache entries invalidated if their stat information

Refer docs for more usage.

iPhone and Chewing Gums

I am sure we all have heard the anecdote of the cobras in Delhi. Concerned about the increase in cobra population, British government announced bounty on dead cobra snake. Initially this worked but eventually enterprising people began breeding corbas for the income. Learning about this, the government scrapped the scheme, which lead to the release of many bredded cobras back into the wild.

Unintended consequences and second order effects in action.

We all know how iPhone disrupted Nokia. But there was another unsuspecting candidate — Wrigley’s chewing gum — which became victim of smartphone revolution.

Visit https://buff.ly/2OokOjV to find how chewing gum sales went down because of Iphone.

What are the second order effects of the current situations we are in?

Crontab Time String Syntax

schtasks is my goto workhorse for scheduleing tasks in windows environment and on mac and raspberrypi, crontab is an excellent choice.

I use crontab a lot in lot of the things I do in raspberry, but I keep forgetting the cron time sting syntax, so here’s one that I have kept as a reference

The Cron time string is five values separated by spaces, based on the following information:

Character Descriptor Acceptable values

1 Minute 0 to 59, or * (no specific value)
2 Hour 0 to 23, or * for any value. All times UTC.
3 Day of the month 1 to 31, or * (no specific value)
4 Month 1 to 12, or * (no specific value)
5 Day of the week 0 to 7 (0 and 7 both represent Sunday), or * (no specific value)

Cron time string Description

30 * * * * Execute a command at 30 minutes past the hour, every hour.
0 13 * * 1 Execute a command at 1:00 p.m. UTC every Monday.
*/5 * * * * Execute a command every five minutes.
0 */2 * * * Execute a command every second hour, on the hour.

Posting it here for my reference. Hope others will find it useful

Skin in The Game

If you take a look at my twitter timeline, many of the retweets are from Nassim Nicholas Taleb.

His work focuses on problems of randomness, probability, and uncertainty, and have read his books multiple times. I was tinking of rereading his last book “Skin in the Game”.

The embedded youtube video is his summary of the book for talks at google. Bookmark this and spend and hour to soak some of this ideas,

Few of the important bits from the book.

• For social justice, focus on symmetry and risk sharing. You cannot make profits and transfer the risks to others, as bankers and large corporations do. You cannot get rich without owning your own risk and paying for your own losses. Forcing skin in the game corrects this asymmetry better than thousands of laws and regulations.
• Ethical rules aren’t universal. You’re part of a group larger than you, but it’s still smaller than humanity in general.
• Minorities, not majorities, run the world. The world is not run by consensus but by stubborn minorities imposing their tastes and ethics on others.
• You can be an intellectual yet still be an idiot. “Educated philistines” have been wrong on everything from Stalinism to Iraq to low-carb diets.
• Beware of complicated solutions (that someone was paid to find). A simple barbell can build muscle better than expensive new machines.
• True religion is commitment, not just faith. How much you believe in something is manifested only by what you’re willing to risk for it.

Have you read this book?