let's say I have a big Matrix X with a lot of zeros, so of course I make it sparse in order to save on memory and CPU. After that I do some stuff and at some point I want to have the nonzero elements. My code looks something like this:

```
ind = M ~= 0; % Whereby M is the sparse Matrix
```

This looks however rather silly to me since the structure of the sparse Matrix should allow the direct extraction of the information.

To clarify: I do not look for a solution that works, but rather would like to avoid doing the same thing twice. A sparse Matrix should perdefinition already know it's nonzero values, so there should be no need to search for it.

yours magu_

I'm not 100% sure what you're after but maybe `[r c] = find(M)`

suits you better?

You can get to the values of `M`

by going `M(r,c)`

but the best method will surely be dictated by what you intend to do with the data next.

The direct way to retrieve nonzero elements from a sparse matrix, is to call `nonzeros()`

.

The *direct way* is obviously the **fastest** method, however I performed some tests against logical indexing on the sparse and its `full()`

counterparty, and the indexing on the former is faster (results depend on the sparsity pattern and dimension of the matrix).

The sum of times over 100 iterations is:

```
nonzeros: 0.02657 seconds
sparse idx: 0.52946 seconds
full idx: 2.27051 seconds
```

The testing suite:

```
N = 100;
t = zeros(N,3);
for ii = 1:N
s = sprand(10000,1000,0.01);
r = full(s);
% Direct call nonzeros
tic
nonzeros(s);
t(ii,1) = toc;
% Indexing sparse
tic
full(s(s ~= 0));
t(ii,2) = toc;
% Indexing full
tic
r(r~=0);
t(ii,3) = toc;
end
sum(t)
```

find function is recommended by MATLAB:

[row,col] = find(X, ...) returns the row and column indices of the nonzero entries in the matrix X.

This syntax is especially useful when working with sparse matrices.

While find has been proposed before, I think this is an important addition:

```
[r,c,v] = find(M);
```

Gives you not only the indices r,c, but also the non-zero values v. Using the *nonzeros* command seems to be a bit faster, but *find* is in general very useful when dealing with sparse matrices because the [r,c,v] vectors describe the complete matrix (except matrix dimension).

Similar Questions

Suppose I create this sparse matrix, where the non-zero elements consist of booleans 'true': s = sparse([3 2 3 3 3 3 2 34 3 6 3 2 3 3 3 3 2 3 3 6], [10235 11470 21211 33322 49297 88361 91470 127422

While solving a machine learning problem using scikit (python) I need to do scaling of scipy.sparse matrix before doing the training using SVM in order to achieve higher accuracy. But its clearly ment

I need to overload [] operator in class Sparse Matrix. This operator must work like in 2D table access. For example tab[1][1], return reference. The problem is I have a vector of elements(struct). tem

I ve got a sparse representation of a square matrix in matlab. I am trying to perform eigen analysis calculating its eigen value and eigen vector n-1 times by removing its time a row and a column. My

I'm working with large sparse matrices that are not exactly very sparse and I'm always wondering how much sparsity is required for storage of a matrix as sparse to be beneficial? We know that sparse r

Suppose I have a sparse matrix Sparstica that is a vertical concatenation of several other sparse matrices. When I type Sparstica(:), I get a list of the nonzero elements. In the left column, will be

I am trying to multiply two large sparse matrices of size 300k * 1000k and 1000k*300k using Eigen. The matrices are highly sparse ~0.01% non zero entries, however there's no block or other structure i

I'm wondering what the best way is to iterate nonzero entries of sparse matrices with scipy.sparse. For example, if I do the following: from scipy.sparse import lil_matrix x = lil_matrix( (20,1) ) x[1

I'm using Python, Numpy and Scipy packages to do matrix computations. I am attempting to perform the calculation X.transpose() * W * X where X is a 2x3 dense matrix and W is a sparse diagonal matrix.

When using pytables, there's no support (as far as I can tell) for the scipy.sparse matrix formats, so to store a matrix I have to do some conversion, e.g. def store_sparse_matrix(self): grp1 = self.g

I read through this page on assigning a sparse matrix. Unfortunately, I do not understand it. Can anyone help me out with an example? For instance, how should I assign the following 10 by 8 sparse mat

I'm working on something which requires me to calculate the elements of a large square matrix repeatedly. The process involves reading data stored in another matrix, and then calculating the matrix el

I am working with an extremely large data set in a sparse matrix format. The data has the filing format (3 tab separated columns, where the string in the first column corresponds to a row, the string

I have a large sparse matrix (lil) implemented in Python with scipy, which comprises of Users on one axis, and songs they played on the other. So each row is a linked list of the songs that user has p

I have to perform this operation: N = A'*P*A The structure of the P matrix is block diagonal while the A matrix is largely sparse (also in a banded structure). The multiplication is performed in blo

I have a very big and sparse matrix, represented as a CSV file (67 GB). Is it possible to load and work with this matrix in Matlab? I can use a 64bit version on a MAC OS computer, 8GB RAM. I have read

Given an arbitrary numpy array (ndarray), is there a function or a short way to convert it to a scipy.sparse matrix? I'd like something that works like: A = numpy.array([0,1,0],[0,0,0],[1,0,0]) S = t

I've found a quite good sparse matrix implementation for c# over http://www.blackbeltcoder.com/Articles/algorithms/creating-a-sparse-matrix-in-net. But as i work in 3d coordinate-system, i need a spar

I am looking for a C library to solve linear and, if possible, nonlinear matrix equation of the form Ax = b. It is important to me, that the packages are not too big and free of charge. Speed does not

I have a csv file with headers like: Given this test.csv file contains sparse matrix: A,B,C,D,E,F,timestamp 611.88243,0,0,0,0,0,0 0,9089.5601,0,864.07514,0,0,0 0,0,5133.0,0,0,0,0 I simp

consider the code below, Am trying to make a Sparse_matrix that contains an array of pointer. In struct Sparse_matrix, Is this a proper way to create a pointer array when size is unknown? What's the

Sorry for rookie question. I'm learning to work with scipy.sparse and I'm out of ideas why this code does not work. The dimensions are correct but the subtraction can not be computed: c=(count_mat[i])

So I have this matrix here, and it is of size 13 x 8198. (I have called it 'blah'). This is a sparse matrix, in that, most of its entries are 0. When I do an imagesc(blah), I get the following image:

I wanted CSR files preferably from matrix market for my OpenCL library, I searched a lot for CSR generators in C but didn't get any. I find matrix market formats comfortable since they have defined th

I have built a small code that I want to use for solving eigenvalue problems involving large sparse matrices. It's working fine, all I want to do now is to set some elements in the sparse matrix to ze

I have a sparse matrix. I need to sort this matrix row-by-row and create another [sparse] matrix. Code may explain it better: # for `rand` function, you need newer version of scipy. from scipy.sparse

I'm populating a sparse matrix in R and have written the update in a for loop but was hoping to get some pointers to make it quicker. Here is some example code: library(Matrix) rowId <- rep(c(101:1

We can construct a sparse matrix from an index and value of non-zero element with the sparseMatrix or spMatrix. Is there any function convert a sparse matrix back to an index and value of all non-zero

If I have a matrix like this A = [1 2; 3 4]; I can use interp2 to interpolate it like this newA = interp2(A,2); and I get a 5x5 interpolated matrix. But what if I have a matrix like this: B = zeros(

I am trying to find the indices of nonzero entries by row in a sparse matrix: scipy.sparse.csc_matrix. So far, I am looping over each row in the matrix, and using numpy.nonzero() to each row to get t

I have a large sparse matrix, and I want to permute its rows or columns to turn the original matrix into a block diagonal matrix. Anyone knows which functions in R or MATLAB can do this? Thanks a lot.

I am doing a text classification task with R, and I obtain a document-term matrix with size 22490 by 120,000 (only 4 million non-zero entries, less than 1% entries). Now I want to reduce the dimension

I have a large scipy sparse symmetric matrix which I need to condense by taking the sum of blocks to make a new smaller matrix. For example, for a 4x4 sparse matrix A I will like to make a 2x2 matrix

This question already has an answer here: Sparse matrices / arrays in Java 11 answers I need to implement a sparse matrix as efficiently memory-wise as I can in Java.I receive a matrix with mor

I want to make a sparse matrix in python. I have the index and value of non-zero elements as a dictionary i.e.: {((1,3),0.0001),(10,4),0.0212)...} which means that value of element (1,3) is 0.0001, (

I would like to multiply single rows of a csr matrix with a scalar. In numpy I would do matrix[indices,:] = x * matrix[indices,:] For csr this raises an exception in scipy. Is there a way to do this

I have an assignment where Im supposed to finish the implementation on a generic sparse matrix. Im stuck on the addition part. The matrix is only going to support numbers so I had it extend Number hop

I have a scipy.sparse.csr matrix and would like to dump it to a CSV file. Is there a way to preserve the sparsity of the matrix and write it to a CSV?

I wish to speed up my machine learning algorithm (written in Python) using Numba (http://numba.pydata.org/). Note that this algorithm takes as its input data a sparse matrix. In my pure Python impleme

I am looking for a good (in the best case actively maintained) C++ matrix library. Thereby it should be templated, because I want to use a complex of rationals as numerical type. The matrices what I a

for sparse matrices i had put a question about its linked representation. there were two lists to be implemented for a nine zero elements of a 5x8 matrix.....one is the coloumn list and another one is

Good Afternoon, I am trying to do: scipy.sparse.dia_matrx(x, shape = (x.size, x.size)) but the resulting shape of the matrix is x.size x 1. Am i doing something wrong? Or did i miss something in the

I'm trying to calculate an expression of the form K = P*C.T*S^-1 (implementation of a Kalman filter) All the involved matrices are sparse, and I would of course like to avoid calculating the actual in

I have to invert a large sparse matrix. I cannot escape from the matrix inversion, the only shortcut would be to just get an idea of the main diagonal elements, and ignore the off-diagonal elements (I

I've been looking to use cuSPARSE to multiply a sparse matrix by a dense matrix. My sparse matrix is in CSC (Compressed Sparse Column Format). cuSPARSE supports conversion from CSC to CSR, and the m

Let's say I have A = [3 0 2; ... 0 0 1; ... 1 1 0] A = sparse(A); The output of which is below: ans = (1,1) 3 (3,1) 1 (3,2) 1 (1,3) 2 (2,3) 1 Question: is there an easy command to generate the fol

I find many similar questions but no answer. For simple array there is multiprocessing.Array. For sparse matrix or any other arbitrary object I find manager.namespace. So I tried the code below: from

I'd like to write a function that normalizes the rows of a large sparse matrix (such that they sum to one). from pylab import * import scipy.sparse as sp def normalize(W): z = W.sum(0) z[z < 1e-6]

I am working on a KNN algorithm for a university assignment and at the moment I'm working on finding the Euclidean distance between each of the training vectors stored as a Scipy lil_matrix (due to th

I have an RDD as such: byUserHour: org.apache.spark.rdd.RDD[(String, String, Int)] I would like to create a sparse matrix of the data for calculations like median, mean, etc. The RDD contains the row_