Author: tyvannguyen

I am a PhD student in CIS program at University of Pennsylvania

Learning Tensorflow

In this writing, I will present some fundamental problems and lessons that I encountered during learning Tensorflow for computer vision problems.  Most of them are from the internet that will come with the original link.

A. Feed the graph

1. For small data, we can use either tf.place holder or files to feed the graph

2. For large data, for example, 150GB, it is recommended to use queued binary file. For more details: https://indico.io/blog/tensorflow-data-inputs-part1-placeholders-protobufs-queues/

 

B. Compact code

While TensorFlow has been developed rapidly, building a pure-Tensorflow graph is not that short as compared to Keras.  A good news is that Keras code using Tensorflow as backend is highly compatible with Tensorflow. Thus, we can definitely use Keras layers as an interface to Tensorflow.

https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html

However, I feel it is complex and hard to use since it is a high-level neural networks API, capable of running on top of either TensorFlowCNTK or Theano. I still cannot figure out how to divide an input tensor to feed different branches of a network. There is an alternative which I feel easier and more Tensorflow-style which is slim. We can get it by putting slim = tf.contrib.slim at the beginning of the code.

 

c. Deconvolutional Layer

The best tutorial that I found:

http://cv-tricks.com/image-segmentation/transpose-convolution-in-tensorflow/

D. Saving and Restoring Models

1. Save Models

 

E. Use only One GPU memory

There is a chance during the testing, your model eats up all GPUs’ s memory so that you cannot run any other job even though just one GPU is working. To avoid this situation, one simple solution is as follows.

 import os
# Set only one GPU available 
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"   # see issue #152
os.environ["CUDA_VISIBLE_EVICES"]="1"    # 1 mean use GPU 1. Set 0 to use GPU 0
 

Then, 
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True, device_count={'GPU':1})) # 1 means use only one GPU 
It will still log out GPU:0, for example, but still use only GPU 1 in  this case.

 

Advertisements

More advice on doing PhD

I am kinda busy and a little lazy these days so I have not rounded up the previous post on this topic. Meanwhile, I unintentionally found another article which is also very much fascinating so I just copied and pasted here and come back for rewriting later.

The article entitled “Graduate School Advice Series: 10 Things You Should Know Before Starting A PhD” which can access here gives 10 suggestions on how to do well on PhD. Here we go.

Graduate School Advice 1: Your #1 Goal Is To Publish Peer Reviewed Articles

Assuming that you want to finish your PhD and become a “doctor”, you have to publish peer reviewed articles. This is the one thing that if you do, you will get your title hands down.

Isn’t the goal to be an expert in your field?

Being an expert without peer reviewed publications equals to being an expert without a PhD2. Click to tweet

It sounds materialistic, specially when compared to this romantic idea of doing science for the progress of human knowledge. Sadly it is how the game is played. This is the rule. Publish or perish.

People are trying to change the game with Open Access, Open Data, Open Science, and writing their results in blogs (if you do not have one, start a science blog right now). These are all very needed initiatives that are changing the scientific game. But still, in order to win the game you need publications (in The Netherlands, where I do my PhD, you need 4).

So the best graduate school advice you could get is write, write, and write some more.

Graduate School Advice 2: The Most Difficult Part Is To Stay Motivated

Since a PhD involves diving very deep into a topic, one might expect that learning very complicated stuff would be the hardest part. If you don’t learn fast enough and well enough, you will not finish your PhD. Right? Not true.

The most likely reason for not finishing a PhD, despite not having publications, is … don’t laugh … quitting your PhD. Click to tweet

What can make you quit your PhD?

It happens usually half way a PhD’s duration. You won’t feel you had enough progress. You will be lost in the middle of an ocean of uncertainty. You will still have in front of you a couple of painful years to endure. The alternative of a bigger pay-check in industry will look really tempting at that point.

That period is called the Valley Of Shit or the Phase 3 of PhD Motivation, the “Crisis of Meaning” . Almost every graduate student goes through this existential crisis.

You just need to realise that it happens to everybody and that you can overcome it. Focusing on constant little progress helps more than evaluating if a bigger goal has been achieved.

Something that worked for me was to start your science blog and share with others my experiences (see how a science blog saved my PhD). It helps to put all your troubles in perspective. If you need help starting a science blog and growing your academic footprint check our videotutorial.

Graduate School Advice 3: You’d Better Finish Your PhD Fast

As you see it sucks to quit your PhD half way. Such a waste of time for going away without a PhD tittle.

Do you know what sucks even more? To spend 4, 5, 6 years and not finish your PhD. Ouch!

I have seen this happening to many people and it has to do with two causes.

One, you realise a bit, just a bit, too late that this is not going to work. Seriously, do you really need 5 years to decide you won’t have enough results and papers to defend your thesis?

Please, evaluate every 6 months if you are still on track, if you are going to make it and correct direction if needed.

In a PhD you should track progress and correct direction regularly, don’t wait till it is too late. Click to tweet

Two, after 4 years you are almost there, you have enough data to write those two last articles and the introduction of your thesis. It feels so close and obvious you are going to get your PhD title that you decide to start a postdoc or a new job. Excellent Wrong choice, Sir!!!!

The stress and pressure to integrate in the new position won’t leave enough room for you to write those last pages. Sure, you are Superman and you are going to write after dinner and during the weekends. I have seen many people failing at this to believe it is a good strategy.

So finish your PhD fast and on time, avoid delaying it. And please, do not start  new job until you really finish your PhD.

Graduate School Advice 4: You Are The Expert In Your Field Of Research

We all regard our supervisors, principal investigators and promotors as a source of infinite knowledge. They are like superhumans.

One common source of frustration is to ask your PhD supervisors for help and realise they know as much as aunt Martha does. If these brilliant guys can’t answer your problems, how are you expected to answer them?

This is why you are here. To answer those questions your bosses cannot answer.

At the end of your PhD you should be the expert on your topic, and not your professor. Click to tweet

You can very well approach your superiors with a problem and propose several solutions. Give arguments for each one. In this context, they might be able to use their scientific instinct, the so called educated guess, to give you a hand.

Graduate School Advice 5: You Won’t Make A (Big) Dent In The Universe

We know you are not in graduate school for the money. Probably you want to contribute to the knowledge of mankind, fix a problem or discover something new. Fair enough.

The sad truth is that for the majority of PhDs, their research will get noticed and used by a handful of other researchers. And that is fine. Most of scientists make big contributions after a lifetime of research, not in a couple years.

The contribution of your PhD to science will be as noticeable as a fart in the middle of a tornado.13 Click to tweet

Then why do it a PhD in the first place? Well, you need to start somewhere and a PhD can give you the tools and skills necessary for achieving higher scientific goals.

Graduate School Advice 6: Key Skills In Graduate School: Reading, Writing, Networking

In kindergarden you learn to read, write, paint and play with other children. Ah those relaxed days when your only worries were choosing the colour of the crayon or if you were going to play hide and seek.

Graduate school is the kindergarden of scientists. You learn again to read, write and interact with others.2 Click to tweet

That’s what you need to learn as a PhD school.

  • Reading: you need to skim over a scientific paper and in a few seconds decide if you should invest the next 30 minutes reading it in depth. If you do, you should easily find what is the novelty of the research presented, if it can be useful for your work, and how does it compare to what you are doing.
  • Writing: in academic papers you need to get to the point. You need to be comprehensive and concise at the same time. You need to be technical yet readable. And if your mother tongue is not English you should work hard not to sound like Google Translate.
  • Networking: although some scientists would love to work in a cave away from the rest of the community, you should interact with other peers. Networking can bring you possible collaborators and chances of writing more papers, your new postdoc position, new ideas, or understanding form fellow PhD students that are also going through the Valley of Shit.

Graduate School Advice 7: Pimp Your Online Reputation And Grow Your Academic Footprint

Traditionally scientists would grow their network and get exposure by publishing papers and attending conference. In this new world, a great piece of graduate school advice is to take care of your online presence.

A PhD student should take care of his online reputation from day 1. Click to tweet

While you should still do these things during your PhD, you could make use of some digital tools to be a better scientist and to grow your online reputation, visibility and academic footprint.

There are 3 key online tools that you can use to grow your online presence as a scientist: a science blog, Twitter, and LinkedIn, aka the social media trinity for scientists.

If you use these three tools you will stop being invisible for Google. People will find you and discover what your research is about.

You can use a science blog to share your opinion about your field of research. You can also share materials like posters and presentations. Or even a description of your papers in simple words.

If you what to start a science blog but don’t know how, check check our videotutorial “Grow Your Academic Footprint With A Science Blog”.

With Twitter you can connect with fellow scientists, share your news and also discover new research related stuff.

LinkedIn can become your online CV, a place where others can easily see your skills, publications and education.

In Next Scientist we are crazy about the digital world, but do not forget to transfer some of these online relations to the physical world. Try to meet face to face with some of your online buddies, either invite them to visit your group, go for a coffee, or arrange that you will meet in a scientific conference.

Graduate School Advice 8: Time Management Rules From The 4 Hour Workweek That Change The Game

If you haven’t read The 4 Hour Workweek yet, buy it, devour it and apply it to your PhD. It is stuffed with great ideas that you can turn into graduate school advice, it will revolutionise the way you see the world.

  • You want to be effective, not just efficient: being efficient at something unimportant is useless. Being effective at finishing important things makes a big difference.7
  • Pareto’s Law 80/20: focus your efforts in that 20% of tasks that bring 80% of the benefits (like writing papers). Remove the 80% of tasks that only contribute to 20% of the results (like revising constantly your time management system).
  • Parkinson’s Law: set tight deadlines, the last minute rush will activate your creativity. If you decide you can do a task in  2 days, guess what? It will take your 2 days to accomplish it. If you would assign 3 hours to it, you would still finish it.6
  • You are scared, so is everybody else: when talking to other people, giving presentations, applying for that position, it is scary, but everybody else would be scared.
  • Have near-impossible goals: these are the goals that motivate you and that are worth working hard and walking the extra mile. When would you work harder? When you have to prepare a poster for a regional meeting or when you have to give a talk at an international conference in New York? I thought so.
Read this post if you want to know more about using The 4 Hour Workweek in your PhD.

Graduate School Advice 9: Deliver Fast And Often, Get Feedback

A great piece of graduate school advice I got when I started was:

At the start of a PhD get some little results fast to boost motivation, don’t go first for big results. Click to tweet

This is great advice because having some small results will a) give you a sense of progress, so your motivation will go up and b) give you something to present (maybe as a poster) and discuss with other scientists.

In your daily work, you should aim at “good enough” and “deliver soon” instead of “perfect delivered in a few more days”. If you deliver intermediate results or a draft of a paper, you have the chance of getting feedback soon and correct your direction if needed.

Bear with me: done is better than perfect.

Don’t wait till you have the perfect figures or till you are not ashamed of the quality of your work. You need to make progress and you need the feedback of your supervisors to do so.

Get rid of your shyness and “move fast, break things, deliver, ship” on a daily basis.

Graduate School Advice 10: Enjoy The Ride

The graduate school advice we shared might sound a bit too harsh. We just want to point out how a PhD really is, so you are not surprised later on. But do not let this discourage you.

Graduate school has many perks that make it a great experience. You will meet interesting people and you will have the chance to explore your own ideas and to be creative.

Remember that you are still a student, so enjoy life like a student. Do not take everything too seriously and make use of your free time.

In some cases you might get a salary or stipend. Isn’t it great to be a student but with money?

For those cases where you don’t get paid or your salary is microscopic, you can easily make a second income in graduate school and enjoy the life of a student.

You have the chance to travel. Get results and present them in conferences. Ask your boss to pay for the trip or apply for a travelling stipend for students. Find collaborators and get them to invite you to visit their lab.

Graduate school is a great time, make good use of your chance of being here.

Now you know how it really is at graduate school. You have almost everything you need to succeed in your new PhD life. What’s missing? You will find in the following posts of these series.

You can also have a look at these great books for PhD students.

Thinking Of Applying to Graduate School? Check These Sample Letters For Graduate School

When applying to graduate school you are going to submit several letters. The goal is to give a good impression and get invited for an interview. You should adapt  to your needs a sample letter that has been successfully used.

[note color=”#efdcde”]Check our letters for graduate school, which include a statement of purpose for graduate school and a recommendation letter for graduate school. You can get the letters for graduate school here.[/note]

Additionally you can get sample letters you will need throughout graduate school, for instance when you are submitting your first scientific paper to a journal or when you are applying to a scientific conference.

Check the letters for graduate school here.

Some Thoughts On Grad School

I went over this post and could not help but copy and paste ^^ it right away as it introduces a lot of sound advice.

Thought #1: Research Trumps All

This is the master thought that most of the other thoughts support. The job of a graduate student is to learn how to do professional-quality research. At the end of your grad school experience you will be judged by the quality and quantity of the research. And that’s basically it. Remind yourself of this truth often. If you’re not making progress on your research, then radically rethink your scheduling priorities.

Thought #1.5: Don’t Let Courses and Quals Distract You From Thought #1

Don’t get too caught up in your courses or qualification exams. Study smart. Do good work. But remember, this isn’t college, and doing well academically is merely a prerequisite for being a successful graduate student — it’s far from the ultimate goal. Keep coming back to your research as priority #1.


Thought #2: Don’t Be a Firefighter

A simple truth: you’ll have more urgent things on your plate than you’ll have time to complete. If you spend your days only putting out one fire after the next as they arrive in your inbox — paper review requests, articles to read, extra experiments to conduct for your advisor — you’ll get very little original research done. This violates thought #1.

This syndrome, fortunately, is easy to avoid. Spend the first 2 -3 hours of the morning doing original work. Only then should you check your e-mail for the first time that day (and let the firefighting begin).

Thought #3: Stick to a Fixed Work Day

The nature of graduate student work is paradoxical. You’ll always feel like you should be working more hours. However, if you add these extra hours, your work output doesn’t increase much. With this in mind, you might as well fix a regular work day (I do 9 to 5:30) and refuse to work beyond these hours (with the obvious exceptions: the night before deadlines, etc.)

Do this, and four things will happen: First, you’ll focus more and get work done faster. Two, you’ll start work earlier which increases its quality. Three, you’ll start turning down time-consuming requests that add little to your career (and be pleased to discover that you’re allowed to say “no”). And four, your stress and guilt will plummet.

Thought #4: Three Projects is Optimal…

Working on one research project at a time is not enough. If you get stuck you can go many weeks beating your head against the wall and getting nothing done. This sucks. More than three projects are too many; quality will suffer and you’ll feel overwhelmed. This also sucks. Juggling three at time seems to be just about right.

Thought #5: …But Don’t Work on More than One Per Day

Within the context of a single day, focus your attention on a single project.

Thought #6: Listen to the Married Graduate Students and Ignore the Unmarried Students Who Live in the Dorms

Students with families have perspective on life and friends outside of the university. They tend to be happy and productive and think sleeping on the futon in your office is childish. They also bathe every day. Which is a nice bonus. The students who are unmarried and living in the dorm have probably escaped, thus far, exposure to the real world in any meaningful form, and because of this they are likely to have a warped sense of personal worth and work habits, and suffer from weird guilt issues. Ignore them.

Thought #7: Promise People Deadlines Then Follow Through

The easiest way to avoid being hassled is to respond to requests with the specific day on which you will complete the work, and then actually follow through. Do this, and people will leave you alone to accomplish things on your own schedule.

Thought #8: Challenge Yourself Once a Month

It’s so damn easy for your research to fall into a rut grooved by short-term decisions based on the question: “What’s the shortest path between here and a new publication?”. Many a graduate student, faced with crafting a job talk after 5 years of work, realizes, with horror, that his “research direction” is weak, jumbled, and uninspiring.

Once a month take yourself out for breakfast and ask: “What is my research mission as a graduate student? And how do I get back there from here?” I imagine this is how lasting careers are founded. (I wish I did this more.)

Thought #9: Don’t Mistake Experience for Smarts

Undergraduates think their graduate student TA’s are smarter than them. Junior graduate students think the senior graduate students are smarter than them. Senior graduate students think their advisors are smarter than them. Sense a pattern? It all comes down to experience. The more time you spend working in a field, the better you get at it, and the smarter you seem to those with less experience. Therefore, when you’re young, don’t get freaked out, and when you’re older, don’t get too impressed with yourself.

Thought #10: Take Days Off

The wonderful thing about being a graduate student is that you don’t have a real job. Your responsibilities are long-term (produce good research) not short-term (answer the phones from 9 to 5). Embrace this fact. Take days off to reward work well done and to unwind. See a movie in the afternoon at least once a month. No one is secretly punching a time clock for you.

Minimalist – again, it is

Yesterday, I met a Nigerian guy – the one that I would never forget for his remarkable saying.

” There are a lot of problems and we don’t need to solve them all to be successful” – he said that when I asked him why he looks so young at his age.

“It’s my young mindset” – he added.

Is practicing exercise or any kind of meditation that keeps him stay young? He told me that that it’s his young mindset.

Indeed, from what I saw, he’s a minimalist guy. He cooked his food using only two stuff: a cooker and a spool which later is used to serve his food. He has the same set of food per meal on every single day. So I asked myself: does that simple daily routine irradicate redundant thoughts and keep him that young?

 

 

Shortest-path Dijktra’s algorithm implementation

This implementation inherits the source code from Geeksforgeeks (matrix implemented). What i did is to implement Dijktra’s algorithm for shortest path problem using linkedlist graph. Souce code can be found here – here

/src/stp_linkedlist : implements graph using linked list

/src/stp_matrix: using adjacent matrix to imploy graph

To run the program, first initialize a graph by editting file /src/graph.txt. This first line of the file is the number presenting the number of nodes in graph. The remaining lines are just a graph representation of the undirected, weighted graph. i.e:

9
0 4 0 0 0 0 0 8 0
4 0 8 0 0 0 0 11 0
0 8 0 7 0 4 0 0 2
0 0 7 0 9 14 0 0 0
0 0 0 9 0 10 0 0 0
0 0 4 0 10 0 2 0 0
0 0 0 14 0 2 0 1 6
8 11 0 0 0 0 1 0 7
0 0 2 0 0 0 6 7 0

Presentation at Unist

Happiness: Can money buy us happiness? 

Link here

“Money does nothing with your happiness, forget it and forget fucking rich people”.- When i was young, so many people told me that claim.

But, don’t get me wrong. I believe that money can get us happiness if we use it properly. In other word, it is the matter of money usage, not money itself.  In this presentation, i cover fundamental definition and terminologies regarding money by views of psychologists and economics. Key factors of money’s contribution to our life are also undercovered  throughout a few minutes of presenting.  All in all, i end up it by expressing on how money can be beneficial to our well-being and happiness.

Study at  Unist

Link here

I’ve been showing up in presentation class which is super funny and interesting. I’ve learned a lot, especially how to stand confidently and comfortably under the spot light. My instructor, Nate is kinda attractive guy who can make you always smile in the class. He is a Canadian, get married and has a young kid. I am really impressed by his bright lecture and his real stories telling his experience about his journey to more than 60 countries worldwide. He is simply brilliant. He knows everything.

This attached file is one of my presentation materials discussing about my University’s graduate studying.

 

Stay – Rihanna-Cello & Piano cover

The following version features a smooth and beautiful piano sound. The cello sound is sensational and touching my soul. In my opinion, those girls are all good – the cello girl is more preferable -, but it would be the best if the piano girl could have played in a more emotional manor.

Besides, i also do love the Africanized version with a colorful and warm vocal feature made by the guy. The cello sound in the latter is not as good as of the above version but in general, that version is worth to enjoy.

[Link]Understanding Derivative in PID Control

Link here

Derivative action at work

John Ziegler and Nathaniel Nichols, the fathers of PID loop tuning, recognized as far back as 1942 that derivative action dampens the control effort. They discovered through trial and error that setting the derivative time to at least half of the process deadtime would slow the controlled response to a step change in the setpoint.

The top trend chart shows a typical first-order process responding to a setpoint change under the influence of a full, three-term PID controller tuned according to the Zeigler-Nichols rules. The middle trend chart shows the same response with the brakes off; that is, with derivative action disabled. The rate at which the process variable climbs after the setpoint change and the degree by which it overshoots the setpoint have both increased.

Ironically, overshoot can be eliminated entirely by leaving the derivative action disabled, decreasing the proportional gain and increasing the integral time. The bottom trend chart shows the process’s setpoint response under the influence of a PI-only controller tuned according to the critical damping rules that call for a 69% smaller proportional gain and an integral time lengthened by a factor equal to the process gain. No derivative action is required.