I have to create a matlab matrix that is much bigger that my phisical memory, and i want to take advantage of the sparsity.

This matrix is really really sparse [say N elements in an NxN matrix], and my ram is enought for this. I create the matrix in this way:

A=sparse(zeros(N));

but it goes out of memory. Do you know the right way to create this matrix?

`zeros(N)`

is creating an NxN matrix, which is not sparse, hence you are running out of memory. Your code is equivalent to

```
temp = zeros(N)
A = sparse(temp)
```

Just do `sparse(N,N)`

.

I'd actually recommend the full syntax of sparse([],[],[],N,N,N).

It's useful to preallocate if you know the maximum number of nonzero elements as otherwise you'll get reallocs when you insert new elements.

Creating an all zeros sparse matrix, and then modifying it is extremely inefficient in matlab. Instead of doing something like:

```
A = sparse(N,N) % or even A = sparse([],[],[],N,N,N)
A(1:N,7) = 1:N
```

It is much more efficient to construct the matrix in triplet form. That is, construct the column and row indices and the nonzero entries first, then form the matrix. For example,

```
i = 1:N;
j = 7*ones(1,N);
x = 1:N;
A = sparse(i,j,x,N,N);
```

Similar Questions

I would appreciate any help, to understand following behavior when slicing a lil_matrix (A) from the scipy.sparse package. Actually, I would like to extract a submatrix based on an arbitrary index lis

I'm populating a sparse matrix in R and have written the update in a for loop but was hoping to get some pointers to make it quicker. Here is some example code: library(Matrix) rowId <- rep(c(101:1

I'm currently working with sparse matrices, and I have to compare the computation time of sparse matrix-matrix multiplication with full matrix-matrix multiplication. The issue is that sparse matrix co

Is there a function in Matlab to assign matrix elements by blocks? For example: Matrix A = 1 1 3 3 3 3 3 8 8 10 1 1 3 3 3 3 3 8 8 10 1 1 4 4 4 4 4 8 8 10 1 1 4 4 4 4 4 8 8 10 1 1 5 5 5 5 5 8 8 10 2 2

Say I have three different 2x2 submatrices, and I want to create a big 6x6 matrix in Matlab in which the three submatrices appear on the diagonal, and all the other values are 0. How can I do this wih

In MATLAB, it is very convenient to create a pentadiagonal sparse matrix using commands like this: I = eye(m); % create identity matrix e = ones(m,1); % create an array of all 1's T = spdiags([e -4*e

I have a quick graph theory question. I have a 13 x 13 adjacency matrix in Matlab. I've already taken just the lower diagonal (this is an undirected graph) and subtracted off the identity matrix (so

I have a large sparse matrix A, and I would like to create a sparse matrix of the 3X3 block diagonals of A. How would I do this? keep in mind that A is very large and sparse, so any methods that use i

To my understanding, numpy.sparse.csr_sparse.dot(other) does multiply other to my sparse matrix from the right: A = numpy.sparse.csr_sparse(something) B = numpy.matrix(something) C = A.dot(B) # C = A*

Why create a matrix of 0's in Matlab? For example, A=zeros(5,5); for i = 1:5 A(i)=exp(i); end

Is there a sparse matrix library that copies the functionality of dense BLAS? I'd want at least: efficient SYR and SYRK (rank-k update) with sparse input (and possibly dense output), option for spars

I'm writing a program that will convert a sparse matrix to Blocked Compressed Row Storage BCRS. I know how to acquire Rowptr, Colind(although not in the code) and A_f. Code: p = 0; for (i = 0; i <

In Matlab, at what point is having a sparse array better than a normal array if I still have a lot of calculations to do on it, and about 25% of the array are non-zeros?

I have a 1034_by_1034 sparse matrix (scipy.sparse.csr.csr_matrix), which basically represents the adjacency matrix of a graph. I want to check if some elements are ones or not. But I found this to be

I find many similar questions but no answer. For simple array there is multiprocessing.Array. For sparse matrix or any other arbitrary object I find manager.namespace. So I tried the code below: from

type(A) <class 'scipy.sparse.csc.csc_matrix'> A.shape (8529, 60877) print A[0,:] (0, 25) 1.0 (0, 7422) 1.0 (0, 26062) 1.0 (0, 31804) 1.0 (0, 41602) 1.0 (0, 43791) 1.0 print A[1,:] (0, 7044) 1.0

I have a scipy sparse matrix, where I need to add multiple rows (in blocks), say 1:30, then 45:50, etc. What is the most efficient way to do this?

Question one: Are there specialized databases to store dense and sparse matrices ? I googled but didn't find any... The matrix in question is huge (10^5 by 10^5) but it's sparse, which means that most

I need a library to solve Ax=b systems, where A is a non-symmetric sparse matrix, with 8 entry per row (and it might be quite big). I think a library that implements biconjugate gradient should be fin

What do you think? What would be faster and how much faster: Doing sparse matrix (CSR) multiplication (with a vector) on the GPU or the CPU (multithreaded)?

I am dealing with a very huge matrix and so wanted to use parallel computing in matlab to run in clusters. Here I have created a sparse matrix using Ad=sparse(length(con)*length(uni_core),length(co

I initialize an empty sparse matrix using S = scipy.sparse.lil_matrix((n,n),dtype=int) As expected print S doesn't show anything, since nothing has been assigned. Yet if I test: print S[0,0]==0 I r

I am using cusp for sparse matrix multiplication. From the resultant matrix i need the max value without copying the matrix from device memory to host memory. I am planning to wrap the resultant matri

Bash can create a sparse array in one line names=([0]=Bob [1]=Peter [20]=$USER [21]=Big Bad John) § Creating Arrays Can JavaScript create a sparse array like this?

I want to pass a sparse matrix to a shared library from MATLAB, do some operation there (written in C), and then return it. I can pass a dense matrix and use, pretty easy. But, I have no idea how to p

I am writing a java program which involves working with a 1058 X 1058 matrix containing float values. This matrix contains many zero values and so I need to store this as a sparse matrix and later use

I read through this page on assigning a sparse matrix. Unfortunately, I do not understand it. Can anyone help me out with an example? For instance, how should I assign the following 10 by 8 sparse mat

I'm trying to write a matrix to sparse matrix conversion program. But, there is a problem in my scan_matrix function.I couldn't figure out where is the problem.According to Nemiver(debugging program)

When trying to directly set the data attribute of a sparse lil_matrix, I encounter very unexpected behavior. Can someone explain what is going on in the following simple example? My particular use ca

Is there a way to compute the power of a sparse matrix in matlab without converting it to a full matrix. If I try b = a^0.5 where a is a sparse matrix, I get the error Use full(x)^full(y).. However

I am trying to find an efficient way to retrieve a list / vector / array of the non-zero upper triangular elements of a sparse matrix in R. For example: library(igraph) Gmini <- as.directed(graph.

I am trying to make my control algorithm more efficient since my matrices are sparse. Currently, I am doing conventional matrix-vector multiplications in Simulink/xPC for a real-time application. I ca

I'm working on a large scale project in which I'm designing a sparse matrix vector application but I'm still working to understand the code. I'm beginning by building the foundation for the applicatio

Suppose I create this sparse matrix, where the non-zero elements consist of booleans 'true': s = sparse([3 2 3 3 3 3 2 34 3 6 3 2 3 3 3 3 2 3 3 6], [10235 11470 21211 33322 49297 88361 91470 127422

I am doing some computations on a sparse matrix of floats in the log domain, so the empty entries are actually -Inf (using -FLT_MAX). I'm using a custom sparse matrix class right now but I am eager

Is there a fast way to remove rows and columns from a large matrix in MATLAB? I have a very large (square) distance matrix, that I want to remove a number of rows/columns from. Naively: s = 12000; D =

I'm currently trying to classify text. My dataset is too big and as suggested here, I need to use a sparse matrix. My question is now, what is the right way to add an element to a sparse matrix? Let's

I want to create an online SQL editor with smart autocompletion. What the best way to do it? If i have empty input field (| - current carriage position), editor must suggest SELECT, INSERT, UPDATE, DE

I have an N x N sparse matrix in Matlab, that has cell values indexed by (r,c) pairs such that r and c are unique id's. The problem is, that after converting this matrix into Python, all of the indic

I am trying to subsample a scipy sparse matrix as a numpy matrix like this to get every 10th row and every 10th column: connections = sparse.csr_matrix((data,(node1_index,node2_index)), shape=(dimensi

I've wondered about this for a while now. When Matlab prints the matrix A, for instance, with A it sometimes appears in scientific notation such as A = 1.0e+03 * 0 0 0.0070 0.0080 0.0030 0.0010 0 0.0

Is there a built-in function in either slam package or Matrix package to convert a sparse matrix in simple triplet matrix form (from slam package) to a sparse matrix in dgTMatrix/dgCMatrix form (from

In Matlab I'm using oracles data provider to query an oracle database. I want to fill a matlab matrix or 2D array with the result. I'm a .NET guy but not strong in Matlab. We have analysts who use Mat

My goal is to combine many sparse matrices together to form one large sparse matrix. The only two ideas I've been able to think of are (1) create a large sparse matrix and overwrite certain blocks, (2

I have (n x n) matrix in Matlab. For example(n=3): A=[1,2,3; 4,5,6; 1,9,9] I want save this matrix to vector (or array) B, but rows should be first. Output: A=[1,2,3,4,5,6,1,9,9] Thank you

For a current project I have to compute the inner product of a lot of vectors with the same matrix (which is quite sparse). The vectors are associated with a two dimensional grid so I store the vector

I've got a scipy.sparse_matrix A and I want to zero-out a decently-sized fraction of the elements. (In the matrices I'm working with today, A has about 70M entries and I want to zero-out about 700K of

I'm trying to find a way to subtract a column of a scipy.sparse matrix from a numpy vector but I can't seem to find a way to do it without changing the shape of the vector. This is what I have so far:

I'm trying to convert the following MATLAB code to Python and am having trouble finding a solution that works in any reasonable amount of time. M = diag(sum(a)) - a; where = vertcat(in, out); M(where,

I'm working with some rather large sparse matrices (from 5000x5000 to 20000x20000) and need to find an efficient way to concatenate matrices in a flexible way in order to construct a stochastic matrix