If I am using the sparse.lil_matrix format, how can I remove a column from the matrix easily and efficiently?

I've been wanting this myself and in truth there isn't a great built-in way to do it yet. Here's a way to do it. I chose to make a subclass of lil_matrix and add the remove_col function. If you want, you can instead add the removecol function to the lil_matrix class in your `lib/site-packages/scipy/sparse/lil.py`

file. Here's the code:

```
from scipy import sparse
from bisect import bisect_left
class lil2(sparse.lil_matrix):
def removecol(self,j):
if j < 0:
j += self.shape[1]
if j < 0 or j >= self.shape[1]:
raise IndexError('column index out of bounds')
rows = self.rows
data = self.data
for i in xrange(self.shape[0]):
pos = bisect_left(rows[i], j)
if pos == len(rows[i]):
continue
elif rows[i][pos] == j:
rows[i].pop(pos)
data[i].pop(pos)
if pos == len(rows[i]):
continue
for pos2 in xrange(pos,len(rows[i])):
rows[i][pos2] -= 1
self._shape = (self._shape[0],self._shape[1]-1)
```

I have tried it out and don't see any bugs. I certainly think that it is better than slicing the column out, which just creates a new matrix as far as I know.

I decided to make a removerow function as well, but I don't think that it is as good as removecol. I'm limited by not being able to remove one row from an ndarray in the way that I would like. Here is removerow which can be added to the above class

```
def removerow(self,i):
if i < 0:
i += self.shape[0]
if i < 0 or i >= self.shape[0]:
raise IndexError('row index out of bounds')
self.rows = numpy.delete(self.rows,i,0)
self.data = numpy.delete(self.data,i,0)
self._shape = (self._shape[0]-1,self.shape[1])
```

Perhaps I should submit these functions to the Scipy repository.

```
def removecols(W, col_list):
if min(col_list) = W.shape[1]:
raise IndexError('column index out of bounds')
rows = W.rows
data = W.data
for i in xrange(M.shape[0]):
for j in col_list:
pos = bisect_left(rows[i], j)
if pos == len(rows[i]):
continue
elif rows[i][pos] == j:
rows[i].pop(pos)
data[i].pop(pos)
if pos == len(rows[i]):
continue
for pos2 in xrange(pos,len(rows[i])):
rows[i][pos2] -= 1
W._shape = (W._shape[0], W._shape[1]-len(col_list))
return W
```

Just rewrote your code to work with col_list as input - maybe this will be helpful for somebody.

I'm new to python so my answer is probably wrong, but I was wondering why something like the following won't be efficient?

Lets say your lil_matrix is called mat and that you want to remove the i-th column:

```
mat=hstack( [ mat[:,0:i] , mat[:,i+1:] ] )
```

Now the matrix will turn to a coo_matrix after that but you can turn it back to lil_matrix.

Ok, I understand that this will have to create the two matrices inside the hstack before it does the assignment to the mat variable so it would be like having the original matrix plus one more at the same time but I guess if the sparsity is big enough then I think there shouldn't be any memory problems (since memory (and time) is the whole reason of using sparse matrices).

Much simpler and faster. You might not even need the conversion to csr, but I just know for sure that it works with csr sparse matrices and converting between shouldn't be an issue.

```
from scipy import sparse
x_new = sparse.lil_matrix(sparse.csr_matrix(x)[:,col_list])
```

For a sparse csr matrix (X) and a list of indices to drop (index_to_drop):

```
to_keep = list(set(xrange(X.shape[1]))-set(index_to_drop))
new_X = X[:,to_keep]
```

It is easy to convert lil_matrices to csr_matrices. Check tocsr() in lil_matrix documentation

Note however that going from csr to lil matrices using tolil() is expensive. So, this choice is good when you do not require to have your matrix in lil format.

By looking at the notes for each sparse matrix, specifically in our case is csc matrix it has the following advantages as listed in the documentation [1]

- efficient arithmetic operations CSC + CSC, CSC * CSC, etc.
- efficient column slicing
- fast matrix vector products (CSR, BSR may be faster)

If you have the column indices you want to remove, just use slicing. For removing rows use csr matrix since it is efficient in row slicing

Similar Questions

cuSparse only has a function api for multiplying a sparse matrix with a dense matrix. How to do multiply operation for two sparse matrices using cuSparse or any other cuda liberary?

I want to partition a QByteArray message efficiently, so this function I implemented take the Bytes, the part I want to extract, and toEnd flag which tells if I want to extract part1 till the end of t

I have got an output using sparse matrix in python, i need to store this sparse matrix in my hard disk, how can i do it? if i should create a database then how should i do?? this is my code: import nl

I have this DateTime: => Fri, 03 Feb 2012 11:52:42 -0500 How can I remove the zone(-0500) in ruby? I just want something like this: => Fri, 03 Feb 2012 11:52:42

I've read several topics, but I'm lost. I'm quite new to this. I want to store huge sparse matrix and have several idea's but can choose between them. Here's my needs: Adjacency matrix of approx. 50

I have this huge sparse matrix A of size 2 million by 10 thousand. I want to index particular 1000 rows (index) from this matrix. If I do B = A(index,:); it takes some time. Is there a better quick

This question already has an answer here: Is there support for sparse matrices in Python? 4 answers I am looking for a solution to store about 10 million floating point (double precision) numbe

I just started to learn to program in Python and I am trying to construct a sparse matrix using Scipy package. I found that there are different types of sparse matrices, but all of them require to sto

I am creating a matrix from a Pandas dataframe as follows: dense_matrix = numpy.array(df.as_matrix(columns = None), dtype=bool).astype(np.int) And then into a sparse matrix with: sparse_matrix = scip

I've found a quite good sparse matrix implementation for c# over http://www.blackbeltcoder.com/Articles/algorithms/creating-a-sparse-matrix-in-net. But as i work in 3d coordinate-system, i need a spar

I wonder how to remove Web Matrix dependecy from web platform installer. For example, I saw screencast how to install wordpress with wpi and there everything was fine without installinb Web Matrix.

I did bind the circle with marker to make view like: But when I try to remove marker from the map, the circle still exists. Hmm, How can I remove marker with circle? Relevant code: function removeM

I need to create a matrix with values from a numpy array. The values should be distributed over the matrix lines according to an array of indices. Like this: >>> values array([ 0.73620381, 0.

How to remove a column from an existing table? I have a table MEN with Fname and Lname I need to remove the Lname How to do it?

I would like to remove the Handling from the Shipping & Handling line of the website, my client charges shipping but never handling. How can I remove this from the front end of the site?

How can I remove RVM (Ruby Version Manager) from my system?

I use some C++ code to take a text file from a database and create a dgcMatrix type sparse matrix from the Matrix package. For the first time, I'm trying to build a matrix that has more than 2^31-1 no

I created a compressed sparse matrix, but while accessing to a positive index it complains that the index is negative: import scipy.sparse as sparse B= sparse.csc_matrix((110111213141516, 25)) B[11011

In SQL, how can I remove the first 4 characters of values of a specific column in a table? Column name is Student Code and an example value is ABCD123Stu1231. I want to remove first 4 chars from my ta

I am using cusp for sparse matrix multiplication. From the resultant matrix i need the max value without copying the matrix from device memory to host memory. I am planning to wrap the resultant matri

I have a sparse matrix W, when I use linalg.pinv(W), it throws some errors: Traceback (most recent call last): File /Users/ad9075/PycharmProjects/bednmf/test.py, line 14, in testNmfRun self.factor =

Well I think the question pretty much sums it up. I have a forward_list of unique items, and want to remove a single item from it: std::forward_list<T> mylist; // fill with stuff mylist.remove_i

I am writing a java program which involves working with a 1058 X 1058 matrix containing float values. This matrix contains many zero values and so I need to store this as a sparse matrix and later use

I have an upper triangular matrix from which I need to extract the non-zero data in an increasing order and corresponding row and column numbers to use in the rest of the matlab program - can someone

I need to store word co-occurrence counts in several 14000x10000 matrices. Since I know the matrices will be sparse and I do not have enough RAM to store all of them as dense matrices, I am storing th

In another post regarding resizing of a sparse matrix in SciPy the accepted answer works when more rows or columns are to be added, using scipy.sparse.vstack or hstack, respectively. In SciPy 0.12 the

I am trying to compute nearest neighbour clustering on a Scipy sparse matrix returned from scikit-learn's DictVectorizer. However, when I try to compute the distance matrix with scikit-learn I get an

let's say I have a big Matrix X with a lot of zeros, so of course I make it sparse in order to save on memory and CPU. After that I do some stuff and at some point I want to have the nonzero elements.

Is there any sparse matrix library that can do these: solve linear algebraic equations support operations like matrix-matrix/number multiplication/addition/subtraction,matrix transposition, get a row

I have a large scipy sparse symmetric matrix which I need to condense by taking the sum of blocks to make a new smaller matrix. For example, for a 4x4 sparse matrix A I will like to make a 2x2 matrix

I have a very large and sparse matrix of size 180GB(text , 30k * 3M) containing only the entries and no additional data. I have to do matrix multiplication , inversion and some similar linear algebra

Might be a very newbie question, but I can't find a way to remove files from my projects in QtCreator. Thanks!

I have seen plenty of answers regarding how to remove leading and/or trailing zeros, and how to remove all zeros from a vector or matrix. What I need to do, though, is only remove some of them. I have

I see with new Eigen 3.2, you can get row, column or even block from a sparse matrix, is there a way to set any of these to 0? Eigen::SparseMatrix<float, Eigen::RowMajor> A( 5, 5 ); A.block(1, 1

I have a very large Scipy sparse (csr) matrix. I can't use M.toarray() since it triggers ValueError: array is too big. Is there a way of saving a Scipy sparse matrix in Python to be read in Matlab? I

Group, I need to remove the '-' from any negative numbers in a column for certain record numbers. However the Sum needs to * by .01 to get the correct format. I tried using replace but it is getting t

Does anyone know how to compute a correlation matrix from a very large sparse matrix in python? Basically, I am looking for something like numpy.corrcoef that will work on a scipy sparse matrix.

I want to make a sparse matrix in OpenCV. How can I do the basic operation for this matrix like: Putting or accessing data from matrix elements. Cheers

I've created a TermDocumentMatrix from the tm library in R. It looks something like this: > inspect(freq.terms) A document-term matrix (19 documents, 214 terms) Non-/sparse entries: 256/3810 Sparsi

I have a fairly large sparse matrix that, I reckon, would occupy 1Gb when loaded into memory. I don't need access to the whole matrix at all times, so some kind of memory mapping would work; it doesn'

I have a matrix A in CSC-format, of which I index just a single column b = A[:,col] resulting in a (n x 1) matrix. What I want to do is: v = M * b where M is a (n x n) matrix in CSR. The result v i

I have a large sparse matrix A, and I would like to create a sparse matrix of the 3X3 block diagonals of A. How would I do this? keep in mind that A is very large and sparse, so any methods that use i

I am trying to remove quotes from something like: Hello so that the string is just: Hello

In SparseSuiteQR, all of the examples I can find use stdin or a file read to create a sparse matrix. Could someone provide a simple example of how to create one directly in C++? Even better, in the CH

I have a 35x2 matrix (randomwords); and I have randomly selected 8 rows (rndm). What I need to do is remove the 8 selected rows from the randomwords matrix and save this new 27x2 matrix under a new va

I am digging through OpenERP web to find how I can remove the exception of date from it. eg I want to insert a date like 12/12/1523. with Current functionality it will show me an error message if i wi

How do I remove the my cart section from the left hand column?

I am working with a large (complex) Hermitian matrix and I am trying to diagonalize it efficiently using Python/Scipy. Using the eigh function from scipy.linalgit takes about 3s to generate and diago

I have a string which contains some trailing control characters: 'ase_Record'#$A#9#9#9' I tried to use StringReplace to remove the control characters, but could not make it work. How can I remove the

I wish to be able to extract a row or a column from a 2D array in Python such that it preserves the 2D shape and can be used for matrix multiplication. However, I cannot find in the documentation how