How To Ignore Rows Where Blank Values Excist Pandas Python

What i'm trying to do is query a Panda DataFrame in order to give me a filtered version of the original one self.waferInfo = pd.read_csv(fileName, index_col= None, na_values=['NA'

Solution 1:

I do not think that your assumptions about the root cause of the problem are correct. See below.

"""
18 5 6 8 2
A  B C   E
D  E T Y P
F  R B A L
"""import pandas as pd
import numpy as np

df = pd.read_clipboard()
print(df)
print("\n")
print(df.dropna())

Output:

1856820AB  C  E  None1  D  E  T  Y     P2  F  R  BA     L


  1856821  D  E  T  Y  P2  F  R  BA  L

If df2.head(5) returns nothing, then it's because df2 is empty, which is not because there are NaN's in your df.

Perhaps

self.waferInfo[(self.waferInfo.FILE_FINISH_TS >= dateBegin) & \
(self.waferInfo.FILE_FINISH_TS <= dateEnd) ]

should be

self.waferInfo.loc[(self.waferInfo.FILE_FINISH_TS >= dateBegin) & \
(self.waferInfo.FILE_FINISH_TS <= dateEnd) ]

I can't say for sure because you haven't provided enough sample data.