I have a large sparse matrix `A`

, and I would like to create a sparse matrix of the 3X3 block diagonals of A. How would I do this? keep in mind that `A`

is very large and sparse, so any methods that use iteration will be slow, and any methods that use some methods that creates full (as opposed to sparse) matrices will take up too much memory.

If I understand correctly, here is some code (see the portions between the `%%%%%%%%%%%`

lines. Below are timing results, which seem reasonable to me, despite the `for`

loop. The only trick is the use of the spalloc function, which you may have to tune for your application.

```
for N= [(3:3:12) (15:600:9000)]
bigsparse = sprand(N,N,0.1);
tic;
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
origSize = size(bigsparse);
diagSize = 3;
numDiags = size(bigsparse,1)/diagSize;
assert(numDiags == floor(numDiags))
bigsparse_diagonals = spalloc(origSize(1), origSize(2), ceil(prod(origSize)*0.1));
for ix=(1:numDiags)-1
ixsCurrent = ix*diagSize+[1:diagSize];
bigsparse_diagonals(ixsCurrent,ixsCurrent) = ...
bigsparse(ixsCurrent,ixsCurrent);
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
fprintf(1,'%5d size --> %6.5f seconds \n', N, toc)
end
```

Timing results (note, it actually takes a lot longer to generate the random test matrix than to do the reformatting):

3 size --> 0.00135 seconds 6 size --> 0.00014 seconds 9 size --> 0.00013 seconds 12 size --> 0.00014 seconds 15 size --> 0.00015 seconds 615 size --> 0.00392 seconds 1215 size --> 0.00874 seconds 1815 size --> 0.01537 seconds 2415 size --> 0.02570 seconds 3015 size --> 0.03595 seconds 3615 size --> 0.05007 seconds 4215 size --> 0.06420 seconds 4815 size --> 0.08690 seconds 5415 size --> 0.10077 seconds 6015 size --> 0.13322 seconds 6615 size --> 0.14923 seconds 7215 size --> 0.17562 seconds 7815 size --> 0.37371 seconds 8415 size --> 0.23060 seconds

Similar Questions

I have a matrix in Matlab where each row looks something like this: 1 3 0.112 5.31275 4.61924 -6.50652 And I want to extract to different matrices according to the value presented in the first column

There are at least two sparse matrix packages for R. I'm looking into these because I'm working with datasets that are too big and sparse to fit in memory with a dense representation. I want basic lin

I ran into the following issue trying to vstack two large CSR matrices: /usr/lib/python2.7/dist-packages/scipy/sparse/coo.pyc in _check(self) 229 raise ValueError('negative row index found') 230 if s

I am constructing a 16x16 matrix, consisting of just letters in MATLAB. I tried for example for i=1:2:3 C(i,2)=char('B'); end to place the letter B in its corresponding place in the matrix, however

Suppose I create this sparse matrix, where the non-zero elements consist of booleans 'true': s = sparse([3 2 3 3 3 3 2 34 3 6 3 2 3 3 3 3 2 3 3 6], [10235 11470 21211 33322 49297 88361 91470 127422

I have a 5000 *5000 sparse matrix with 4 different values. I want to visualise the nonzero elements with 4 different colors such that I can recognise the ratio of this values and the relationships bet

Anyone know of any good sparse matrix library? I need it for doing kronecker products and multiplication on large sparse matrices (10,000 x 10,000). Right now we are using R, which handles them prett

I need to construct a large NxN sparse band matrix A with N = 570*720 = 410400 (# of image pixels). Mathematically, A(m,n) = C1 * exp(-|m-n|^2); m = 1:N, n = 1:N Basically its a Gaussian function eva

Using the Matrix package I can create a two-dimensional sparse matrix. Can someone suggest a package that would allow me to create a multidimensional (specifically a 3-dimensional) sparse matrix (arra

I am trying to factorize very large matrixes with the python library Nimfa. Since the matrix is so large I am unable to instanciate it in a dence format in memory, so instead I use scipy.sparse.csr_ma

Basically, I am just trying to do a simple matrix multiplication, specifically, extract each column of it and normalize it by dividing it with its length. #csc sparse matrix self.__WeightMatrix__ = s

I am converting from a scipy sparse matrix to a dense matrix and adding that to an ndarray using a += operator and I am getting a broadcast error. The ndarray has a shape (M,) while the dense matrix h

I have lots of vectors like this one below, very sparse, lots of 'NaN'. What I intend to do is to extract the valid number out of this vector, and put them into a separate vector with no 'NaN' values.

What is the state of the art for fastest linear solver for sparse, positive semi definite and strictly diagonally dominant matrix with N varies from ~700 to ~3000, and about a 1/16 of the matrix is no

I am a total beginner in MATLAB and I hope to find some help here. I have some model prediction results for 80 individuals alltogether in one large matrix. I need to extract the data for each individu

Matlab question. I have this RGB matrix of a set of different pixels. (N pixels => n rows, RGB => 3 columns). I have to calculate the minimum RGB distance between any two pixels from this matrix

I'd like to write a function that normalizes the rows of a large sparse matrix (such that they sum to one). from pylab import * import scipy.sparse as sp def normalize(W): z = W.sum(0) z[z < 1e-6]

I am working with an extremely large data set in a sparse matrix format. The data has the filing format (3 tab separated columns, where the string in the first column corresponds to a row, the string

I am dealing with a very huge matrix and so wanted to use parallel computing in matlab to run in clusters. Here I have created a sparse matrix using Ad=sparse(length(con)*length(uni_core),length(co

I have this huge sparse matrix A of size 2 million by 10 thousand. I want to index particular 1000 rows (index) from this matrix. If I do B = A(index,:); it takes some time. Is there a better quick

I have a matrix A in CSC-format, of which I index just a single column b = A[:,col] resulting in a (n x 1) matrix. What I want to do is: v = M * b where M is a (n x n) matrix in CSR. The result v i

I need to overload [] operator in class Sparse Matrix. This operator must work like in 2D table access. For example tab[1][1], return reference. The problem is I have a vector of elements(struct). tem

I'm having trouble resizing a matrix - the set_shape function seems to have no effect: >>> M <14x3562 sparse matrix of type '<type 'numpy.float32'>' with 6136 stored elements in LInk

I am using the C++ Eigen 3 library in my program. In particular, I need to multiply two Eigen sparse matrix and store the result into another Eigen sparse matrix. However, I noticed that if the some e

I'm involved in the resolution of a system of the type Ax = b, where A is a square sparse matrix, x is the vector of the unknows (I have to compute it) and b is a vector of all zeros excpet for the la

I am trying to create a large sparse matrix, 10^5 by 10^5 in R, but am running into memory issues. > Matrix(nrow=1e5,ncol=1e5,sparse=TRUE) Error in Matrix(nrow = 1e+05, ncol = 1e+05, sparse = TRUE)

I have a number of scipy sparse matrices (currently in CSR format) that I need to multiply with a dense numpy 1D vector. The vector is called G: print G.shape, G.dtype (2097152,) complex64 Each spars

I have got an output using sparse matrix in python, i need to store this sparse matrix in my hard disk, how can i do it? if i should create a database then how should i do?? this is my code: import nl

I'm looking for a matrix / linear algebra library in Java that provides a sparse matrix that can be written to concurrently from different threads. Most of the libraries I've come across either do not

Given a vector of the counter-diagonals of a matrix in matlab, is there an easy way to reconstruct the matrix? For example, given x = [1 2 3 4 5 6 7 8 9] is there any easy way to reconstruct it to th

If have an array (5000x1 double) in matlab workspace. I put the 'from workspace' block in simulink window for input of another block, But when run the program this error occurred: Invalid matrix-form

I am implementing a sparse matrix based on the Stack class, and I'm getting the following error: Sparse.java:6: Sparse is not abstract and does not override abstract method pop() in Stack public clas

What is the fastest way to convert 1/0 sparse matrix to 0/1 sparse matrix without using todense() method? Example: Source matrix looks like: matrix([[1, 1, 0, 0, 0, 0, 0, 0, 1, 1], [1, 1, 0, 0, 1, 1,

I have a very large Scipy sparse (csr) matrix. I can't use M.toarray() since it triggers ValueError: array is too big. Is there a way of saving a Scipy sparse matrix in Python to be read in Matlab? I

I need to calculate the sum of two diagonals in a matrix in C++, I already have a solution for that but I must be dumb because I cant understand what it is doing, so I would like to know if there is a

In SparseSuiteQR, all of the examples I can find use stdin or a file read to create a sparse matrix. Could someone provide a simple example of how to create one directly in C++? Even better, in the CH

I am using CHOLMOD in SuiteSparse to factor an N by N large band-diagonal matrix that is relatively sparse, ie, it contains only a few diagonals that are non-zero. The sparsity of the matrix is set by

I am using MATLAB. I have very large sparse matrices, and I want to perform a logical or bsxfun on each column of this matrix. There is a single for loop where in it is a single operation of logical t

Does Matlab support efficient operations on large sparse tensors? More specifically: Is there an elegant way, similar to sparse, of loading and storing a sparse tensor? As far as I can understand, sp

I have the following matrix which I believe is sparse. I tried converting to dense using the x.dense format but it never worked. Any suggestions as to how to do this?, thanks. mx=[[(0, 2), (1, 1), (2,

There have been a few questions inquiring about general math/stats frameworks for Scala. I am interested in only one specific problem, that of solving a large sparse linear system. Essentially I am l

How to create a 2d sparse matrix in a MEX-file written in C. After creating the matrix how to access the elements individually like in C , say mat[i][j]? I tired using mxCreateNumericArray function bu

I am building a nxn matrix in matlab with the following code: x = linspace(a,b,n); for i=1:n for j=1:n A(i,j) = x(j)^(i-1); end A i b(i) = (1/i)*x(n)^i - (1/i)*x(1)^i; end I am testing it with a=1 b=

I have two sparse matrices, m1 and m2: > m1 <- Matrix(data=0,nrow=2, ncol=1, sparse=TRUE, dimnames=list(c(b,d),NULL)) > m2 <- Matrix(data=0,nrow=2, ncol=1, sparse=TRUE, dimnames=list(c

i'm running the following code, where M is a ~200,000 by ~200,000 sparse matrix and points is ~200,000 by 2 matrix inds=sub2ind(size(M),points(:,1),points(:,2)); M(inds)=M(inds)+1; the problem is tha

I have a large matrix, let's call it A, which has dimension Mx3, e.g. M=4000 rows x 3 columns. Each row in the matrix contains three numbers, eg. [241 112 478]. Out of these three numbers, we can cons

This question already has an answer here: Matlab: add vector to matrix 2 answers I am trying to add a matrix with another matrix in MATLAB. The first matrix looks like this: 0.0963928845397177

I was using cern.colt.matrix.* lib for sparse matrix calculations ..but it seems that I keep running into this error: Exception in thread main java.lang.IllegalArgumentException: matrix too large I

Are there any storage optimized Sparse Matrix implementations in C#?

I would like to get the minimum nonzero values per row in a sparse matrix. Solutions I found for dense matrices suggested masking out the zero values by setting them to NaN or Inf. However, this obvio