Skip to content Skip to sidebar Skip to footer

How To Access Index In A Pandas Hdstore (pytables)

I have a large HDFStore with a multi-index. How can I get a hold of one of the index levels? I see I can access the colindex like so: store._handle.root.data.table.colindexes But

Solution 1:

The multi-index example from the docs

In [21]: index = MultiIndex(levels=[['foo', 'bar', 'baz', 'qux'],
                                    ['one', 'two', 'three']],
                                    labels=[[0, 0, 0, 1, 1, 2, 2, 3, 3, 3],
                                    [0, 1, 2, 0, 1, 1, 2, 0, 1, 2]],
                                    names=['foo', 'bar'])

In [22]: df_mi = DataFrame(np.random.randn(10, 3), index=index,columns=['A', 'B', 'C'])

In [23]: df_mi
Out[23]: 
                  A         B         C
foo bar                                
foo one    0.385666 -0.0139801.787451
    two   -0.1907280.574169 -0.115581
    three  0.8233081.8452040.603430
bar one   -0.863913 -1.544945 -0.598322
    two   -0.9414841.080205 -0.086216
baz two   -0.5106570.2057811.747204
    three -1.322135 -1.131720 -0.838862
qux one    0.346890 -0.9057620.348206
    two   -1.378711 -0.7517012.229486
    three  0.120956 -0.535718 -0.344610

Store in table format (note that in 0.13 you will pass format='table')

In [24]: df_mi.to_hdf('test.h5','df_mi',table=True,mode='w')

To retrieve a single column. The MI are stored as columns!

In [25]: store = pd.HDFStore('test.h5')

In [26]: store.select_column('df_mi','foo')
Out[26]: 
0    foo
1    foo
2    foo
3    bar
4    bar
5    baz
6    baz
7    qux
8    qux
9    qux
dtype: object

In [30]: store.select_column('df_mi','bar')
Out[30]: 
0      one
1      two
2    three
3      one
4      two
5      two
6    three
7      one
8      two
9    three
dtype: object

In [31]: store.close()

Post a Comment for "How To Access Index In A Pandas Hdstore (pytables)"