TextRank is an algorithm based upon PageRank for text summarization. In TextRank, the vertices of the graph are sentences, and the edge weights between sentences denotes the similarity between sentences.

Use the following steps, we can extracte important sentences from a set of documents.

1. Sentence identification: transfer the documents into sentences
2. Tokenization: Split each sentence into a set of words
3. Similarity calculation: Calculate the similarity between sentences
4. Build sentence graph: build a graph of the sentences
5. TextRank: score the sentences via pagerank

Sentence identification

We can use nltk’s included Punkt module to get sentences from a document.

When you run python from shell, you may encounter the following error:

import: unable to open X server `' @ error/import.c/ImportImageCommand/368

Double check you have the proper shebang line in the beginning of your python script:

#!/usr/bin/env python

Once your set the shebang line, you can run your python script as :

./file.py

We list the top json related operations which include load, loads, dump, dumps and pretty-print json.

Create a json file from a python dictionary

We can easily store a python dictionary into a json file using the json dump method. In the following code, we first define a dictionary, then transfer that dictionary into a json file:

The content of my.json file looks like this:

How to pretty-print JSON?

You can run python with the json.tool option to build a more readable json file: prettyprint json.

There is often a need to call shell command from python directly. In this post, I use examples to show how to run external shell commands in python.

The recommended way to call shell command from python is using the subprocess library. See the following example:

In this post, I use example to show how to run shell command in a python program. However, this method has some problems as the output is buffered into memory.

We need to print out the shell output on the fly if the size is too large.

Python is a popular programming language that can be used to conduct almost any project. When you learn python, you may come up with different questions regarding various tasks such as file processing, list, dict usage, database, time, url, et al. In this tutorial, we give clean solutions to some of the most frequently problems you may encounter when you learn python.

1. File related questions
   How to check whether a file exists using Python?
   How to check whether a path is a file?
   How to make sure an directory exist?

We share clean solutions for some of the most popular questions you may encounter when you process files or directories using Python.

1. How to check whether a file exists using Python?
   import os.path os.path.exists(file_path)

   1 2	import os.path os.path.exists(file_path)

2. How to check whether a path is a file?

   You can use the os.path.isfile function:
   It returns True if path is an existing regular file. This follows symbolic links, so both islink() andisfile() can be true for the same path.

   import os.path os.path.isfile(fname)

   1 2	import os.path os.path.isfile(fname)

3. How to make sure an directory exist?

Python is almost a-must-have skill for data scientist, as you can see many data scientist positions require python programming skills. This post introduces some of the most popular python modules for data science. They are widely used to conducted projects related to data mining and machine learning, and normal data analysis.

1. SciPy. SciPy (pronounced “Sigh Pie”) is a Python-based ecosystem of open-source software for mathematics, science, and engineering. It provides a wide range of algorithms and mathematical tools for data scientist.

2. NumPy. NumPy is the fundamental package for scientific computing with Python.

There are so many python libraries, but sometimes we don’t know which one can be used to solve a certain problem. In this post, I will describe some of the most popular python libraries for different tasks.

1. Django: It is most famous framework to develop web applications. It is a high-level Python Web framework that encourages rapid development and clean, pragmatic design.

2. IPython. IPython is a command line shell for interactive computing with many useful enhancements over the “default” Python interpreter.

   IPython Notebooks are a great environment for scientific computing: Not only to execute code,

Given an array and a value, remove all instances of that value in place and return the new length.

Do not allocate extra space for another array, you must do this in place with constant memory.

The order of elements can be changed. It doesn’t matter what you leave beyond the new length.

Example:
Given input array nums = [3,2,2,3], val = 3

Your function should return length = 2, with the first two elements of nums being 2.

Hint:

1. Try two pointers.

Given a non-negative integer n, count all numbers with unique digits, x, where 0 ≤ x < 10n.

Example:
Given n = 2, return 91. (The answer should be the total numbers in the range of 0 ≤ x < 100, excluding [11,22,33,44,55,66,77,88,99])

Hint:

1. A direct way is to use the backtracking approach.
2. Backtracking should contains three states which are (the current number, number of steps to get that number and a bitmask which represent which number is marked as visited so far in the current number). Start with state (0,0,0) and count all valid number till we reach number of steps equals to 10n.