Over sampling

from imblearn.over_sampling import RandomOverSampler

ros = RandomOverSampler(random_state=42)

x_ros, y_ros = ros.fit_resample(XY.iloc[:, :-1], XY.loc[:, 'label'])
print('Original dataset shape:', Counter(y))
print('Original dataset shape:', Counter(y_ros))

XY = pd.concat([x_ros, y_ros], axis=1)

Using pipenv in Jupyter Notebook

Ipykernel is the Python execution backend for Jupyter. Jupyter Notebook automatically ensures that Ipykernel is available but if I want to use a kernel in a virtual environment, I need to install it manually. (Read)

1. Install ipykernel in the project folder where my pipfile is located.

pipenv install ipykernel

2. Start the virtual environment.

pipenv shell

This will launch the virtual environment of the project.

(my-virtualenv-name) $

3. Install Python kernel with a name

python3 -m ipykernel install --user --name=my-virtualenv-name

In my case, I used ml-basics for my virtual env name.


You can now see the kernel name ml-basics in the kernel option.


Source: SlackOverflow


conda command not found after installing oh-my-zsh

conda command not found

I recently installed oh-my-zsh to avoid my mistakes I have been constantly making while using git and improve my command line interface experience.

oh-my-zsh was great but It suddenly started not recognizing conda command.

Screen Shot 2018-02-23 at 2.24.56 AM

Naturally, I got a problem with using conda virtual environment.

Screen Shot 2018-02-23 at 2.17.32 AM

Add Path to ~/.zshrc file

It seems like this new theme in zsh does not have path to conda so does not know where to look when conda command comes. As suggested here, I found the right path from .bash_profile and pasted it to .zshrc file.

So first, open your .bash_profile file by running following command.

open ~/.bash_profile

You will see the PATH automatically added by Miniconda in my case. If you installed Anaconda, you will see Anaconda instead. Copy this two lines.

# added by Miniconda3 installer
export PATH="/Users/mkang/miniconda3/bin:$PATH"

Go back to the terminal and open .zshrc file via vim and paste the above lines somewhere.

vim ~/.zshrc

source the file or open a new terminal to make the change in effect.

source ~/.zshrc

Now zsh understands conda command!

Screen Shot 2018-02-23 at 2.43.36 AM

Using different Jupyter kernels for different Conda environments

While using new Python 2 Kernel for Jupyter Notebook installed last time (Previous post), I found I cannot use Python libraries I had in my Python 2 in my local. Apparently, Python 2 environment used in the new Python 2 Kernel was not the same Python 2 environment that I have been using so far.

To resolve this issue and avoid confusion, I separated python environments using Conda and tie a specific environment to each Jupyter Kernel.

The first step is obviously creating a python environment (Later I will explain how I did it but here is the basic tutorial for this). But unfortunately, running Jupyter notebook from an environment does not mean the notebook will run in the same environment. It uses the default Conda environment instead of specific environment you are running the notebook from.

Installing nb_conda

I had to install nb_conda, which is an extension to provide Conda environment and an access to the associated packages from within Jupyter (Ref). Detailed info can be found here on GitHub.

This is how you install nb_conda.

$ source activate py2
(py2) $ conda install nb_conda

Checking Kernel from Jupyter

Run Jupyter Notebook after activating your environment. You can see now your Conda environment can be found in the kernel list. You can also check which kernel you are running the current notebook with from the top right.

(py2) $ Jupyter notebook

conda kernel found in jupyter

Jupyter notebook running a wrong Python kernel (Python 2 vs. Python 3)

I don’t know exactly what caused this and when, but I started to get an error while using Jupyter Notebook. The error was basically saying I can’t use Python 2 syntax even though I was in Python 2 Kernel.  I also couldn’t use packages installed in Python 2 environment such as Pandas.

jupyter notebook error
print syntax error. It requires me to use Python 3 syntax while I am in Python 2 kernel.

The printed system version tells me I am using Python 3.6 instead of Python 2.7.

jupyter notebook version check
Why Python 3 in Python 2 kernel?

After a quick research, I figured that I did not have right kernelspec for Python 2 and Python 3 and Jupyter Notebook automatically found and used Python 3 as a default (Source: GitHub discussion). As suggested here, I set up a new kernel for Python 2.

Minkyungs-MacBook-Pro:~ mkang$ sudo python2 -m ipykernel install
Installed kernelspec python2 in /usr/local/share/jupyter/kernels/python2

Now I have two kernels both for Python 2 and Python 3.

 In the file, I have language python for Python 2.
"display_name": "Python 2",
"language": "python",
"argv": [

My guess is that something went wrong while in the automatic migration from Ipython Notebook to Jupyter Notebook as explained here (http://jupyter.readthedocs.io/en/latest/migrating.html?highlight=kernelspec) but I am not 100% sure.

Core values to have for another successful year


core valuesI learn so many things every day from my work: persistence that I see from my manager, work ethics that I learn from my colleagues, passion that I feel from leadership, and so much more. What resonated with me recently was what Logan, the CEO of my company, said in one of our training sessions. Starting a new year, he shared 6 core values he wishes us to have for another successful year in 2018.

There are countless articles and lectures about ‘key values for success’, ‘tips to succeed’, and ‘X ways to inspire employees’, but I guess they become more compelling and inspiring only when they are said by someone who you know that they actually went through hard times and learned by their firsthand experience. Though only about an year, as I have seen him making hard decisions, prospering, succeeding, and failing, I could feel that the list is from his own experiences, not from some arbitrary articles. I also know that there are some items that I really need to improve myslef.

Okay, so here are the six core values form Logan and what he actually said in the session:

1. Courage

When I think about courage, I think about sort of questing into the unknown. There is a Spanish saying that You always need to be able to make a decision with only 70% of the information”. It feels scary because you wish you had 100% of the information, but the reality is that perfection stifles progress very often. So, having the courage to stand up, speak, take a chance, and believe in yourself is an important thing that I am hoping to see and demonstrate in all of you along with your journey in this year of 2018.

2. Empathy

Courage without empathy is being arrogant. Being able to be courageous is one thing but not listening to others is another thing. So, making sure that we are able to listen to each other, making sure that we are able to listen to our customers’ needs and being empathetic to what their needs are for each of your roles and each of personas is another important attribute that we really need to have. Just think about: what are some examples of making sure that we are listening? Making sure that we are caring about the fact that our colleague who had one hell of a week because of three fires that they had to put out? We figure out how to be a little bit more available to them for the week because you know that when you have a bad week, they are going to be able to come back for you in return.

3. Unity

Being able to work together in a unified fashion. For me personally, I am trying to understand how to drive more elegant communications across all of us. I think there is no question at that if we are going to be able to succeed as a team, all of the cylinders must be running. We can’t have a couple of cylinders running and a couple of cylinders not running. So, the goal here is to make sure that we are raising those voices and that we are able to listen, sympathize, and come up with solutions.

4. Persistence

In entrepreneurship, nothing is going to work the first time. Nothing. Nothing. Nothing. Do you know how many times I hit my head against the wall to try to get Aquicore working? I can’t even count the number of times that I have tried and I have failed. This is not a game of perfection. This is not a game of 100% successful times. You will try and you will fail. We will try and we will fail. But persistence is what is going to prevail and win the game. When I was rowing (I don’t know how well you are familiar with my background, I used to be in a rowing team in college), the team that I rowed with was a very under-funded team. It was a team that had 60 people and it had enough money for a 4 person team. But regardless of that circumstance, we were still in the national rank every year. We beat Purdue, we beat MIT, and we beat all of those Yalies and IVYs. All that took was persistence: It took being willing to wake up in the morning. It took being willing to plow through that frustration that you get when you were sitting in the moment where you feel like you can’t solve the problem. I personally never forget that you have the team around you to support you and you have that core value to support you when you get into that moment. 

5. Improvement

I would like to emphasize the clinging desire to improve. I think a lot of us, and I will admit this, probably got little comfortable in 2017. We found our message, we found our system, and teams are doing its thing. But we kind of realize that there was something that was in debt. We knew there were pending needs well beyond what we were achieving. So, being able to recognize that and push forward with our improvement is essential. I think the Product team is a great example where we were trying to create a system around improving on a regular basis – through the use of retrospective and through the use of introspection, listening to yourself and hearing what is wrong.

6. Urgency

No entrepreneur story starts without urgency, unfortunately, or fortunately for that matter. It is what makes it exciting and what makes it stressful. The reality is that there is an urgent need for us to be able to accomplish each of these goals within the timelines.

It is the attitude to what we prize that we admire.

It was not the hero I admired but the reflection from his epaulet or helmet. It is nothing (for us) permanently inherent in another, but his attitude or relation to what we prize that we admire. – Henry David Thoreau.

I recently watched a spiritually inspiring talk on YouTube by a Korean novel writer, Chulhwan Lee. If I translate the title, it is “Ways to win someone’s heart”. It sounds like introducing some flirting tactics, but it is in fact about building healthy relationships by understanding the human nature and what matters to us human.

What stood out to me was a story about an actor. The story was like this: Lee (the speaker) once visited a theater for his actor friend. While waiting for his friend to finish his practice, he saw a guy on the stage repeating the same action over and over again: throwing a tablecloth to the table. He figured that the guy was an actor who did not have any lines in the play. He just had that small part in the opening where he throws the tablecloth. Later, Lee went to see the show. There, he saw the guy proudly walked into the stage, throwing the tablecloth. The tablecloth flew like a magic carpet and perfectly settled on the table. The audience was impressed and applaud him.

Well, it must not have been the importance of the role the audience admired. Rather, it was the attitude the actor had towards his role even if it was small and insignificant. He treated his role as if it was the most important one in the play and himself like the most talented actor in the world. He took his part seriously and tried to make it perfect instead of being resentful about being given a small role or not having any lines in the play. What won the audience’s hearts is his attitude towards what he believes is worth.

We (or more specifically, I) sometimes grumble about not having a significant role, not having an extraordinary talent, not having a better situation, and more and more. However, what makes me special at the end will be not what I have now or inherently but the attitude towards what I believe is worth.

For the record, I here keep the original script of the part that I liked in the talk.

이철환 작가의 세바시 강연 “어떻게 사람의 마음을 얻을 것인가” 중.

제가 만났던 멋진 배우를 소개하겠습니다. 후배를 만나러 극단으로 갔는데 배우들은 무대 위에서 연습하지 않고 무대 아래 의자에 앉아 대사 연습을 하고 있었습니다. 그런데 한 남자가 무대 위에서 굉장히 단순한 동작을 반복하고 있어요. 테이블 위에 테이블보를 펴는 것. 참 쉬운 일이잖아요? 그 일을 거의 두 시간이 넘도록 계속 반복하고 있었습니다. 저는 그가 누구인지 궁금했습니다. 대사 연습을 마친 후배를 통해 저의 궁금증을 풀 수 있었습니다. 그에게는 대사가 한 마디도 없었던 것입니다. 그러니 그는 대사 연습이 필요 없었던 것이지요. 저같으면 하지 않을 것 같은데요. 그의 생각은 달랐던 것 같습니다.

여러날 후, 저는 후배의 공연을 보러 갔습니다. 연극 시작부분에 다시 이 배우가 걸어나왔는데 그의 어깨 위에 가지런히 테이블보가 올려져 있었습니다. 그는 관객을 향해 씩 웃더니 춤을 추는 것처럼 자기 어깨 위에 있는 테이블보를 제법 멀리 떨어져 있는 테이블을 향해 힘껏 던졌습니다. 그 테이블보는 마치 마법의 양탄자처럼 테이블에 사뿐히 내려 앉았는데요, 네 개의 모서리에 자로 잰 듯 정확히 나누며 내려앉았습니다. 그 순간 관객석에서 와 하는 탄성이 나왔습니다. 저 또한 감동받았고요. 그런데 저는 이 배우의 상황을 알고 있으니 마음이 참 뭉클했습니다. 관객을 감동시키는 것은 배역이 아니라 배역을 향한 진정성이라는 것을 그는 알고 있었던 것입니다.

헨리 데이빗 소로우가 말했죠. “사람들을 감동시키는 것은 그의 재능이 아니다. 사람들을 감동시키는 것은 가치있는 것에 대한 그의 태도이다. ” 바꿔서 말씀드리면 사람의 마음을 얻기 위해 필요한 것은 우리의 재능이 아니라는 것입니다. 가치있는 것에 대한 태도, 우리의 마음가짐, 우리의 진정성, 그것이 훨씬 더 중요하다는 것입니다. 이 배우는 자기 스스로를 훌륭한 배우로 인정했던 배우입니다. 그러니 어쩌면 보잘것 없다고 느껴지는 그 작은 배역에 최선을 다해 자신의 진정성을 보여줬죠. 그리고 관객들의 마음을 얻었습니다.

나를 인정한다는 것은 무엇일까요. 나를 인정한다는 것은 나를 기다려주는 것이라고 생각합니다. 지금 내 모습이 보잘것 없어도 언젠가는 멋진 사람이 될거라고 믿으며 나를 기다려 주는 것. 지금 내 눈 앞에 도저히 넘을 수 없을 것 같은 장벽이 있다해도 언젠가는 반드시 그 장벽을 넘을 수 있을 것이라 믿으며 나를 기다려주는 것. 지금 내 앞에 도저히 고칠 수 없을 것 같은 습관이나 상황이 있어도 언젠간 그것이 나아질 것이라 믿으며 나를 기다려주는 것. 저는 그것이 나를 인정하는 것이라고 생각했습니다.