easy sampling of vectors from a sparse matrix, and creating a new matrix from the sample (python)
This question has two parts (maybe one solution?):
Sample vectors from a sparse matrix: Is there an easy way to sample vectors from a sparse matrix? When I'm trying to sample lines using random.sample I get an TypeError: sparse matrix length is ambiguous.
from random import sample
import numpy as np
from scipy.sparse import lil_matrix
K = 2
m = [[1,2],[0,4],[5,0],[0,8]]
sample(m,K) #works OK
mm = np.array(m)
sample(m,K) #works OK
sm = lil_matrix(m)
sample(sm,K) #throws exception TypeError: sparse matrix length is ambiguous.
My current solution is to sample from the number of rows in the matrix, then use getrow(),, something like:
indxSampls = sample(range(sm.shape), k)
sampledRows = 
for i in indxSampls:
Any other efficient/elegant ideas? the dense matrix size is 1000x30000 and could be larger.
Constructing a sparse matrix from a list of sparse vectors: Now imagine I have the list of sampled vectors sampledRows, how can I convert it to a sparse matrix without densify it, convert it to list of lists and then convet it to lil_matrix?
I want to remove a (*) asterisk from my matrix and write out that matrix to a text file and the remaining elements will be concatenated to each other without a space or any kind of delimiters. I wrote
It seems to me that in SAGE the only difference between creating a dense matrix and a sparse matrix is by the flag passed to the constructor (sparse = True). In particular, this means that if I want
I have never use R ,but now I need import a sparse matrix to do association rule in R My import data is a sparse matrix like this: i j x 1 2 3 1 2 3 5 1 3 3 1
In another post regarding resizing of a sparse matrix in SciPy the accepted answer works when more rows or columns are to be added, using scipy.sparse.vstack or hstack, respectively. In SciPy 0.12 the
I have a large scipy sparse symmetric matrix which I need to condense by taking the sum of blocks to make a new smaller matrix. For example, for a 4x4 sparse matrix A I will like to make a 2x2 matrix
I am using Scipy to construct a large, sparse (250k X 250k) co-occurrence matrix using scipy.sparse.lil_matrix. Co-occurrence matrices are triangular; that is, M[i,j] == M[j,i]. Since it would be high
What is the best way to efficiently remove columns from a sparse matrix that only contain zeros. I have a matrix which I have created and filled with data: matrix = sp.sparse.lil_matrix((100, 100)) I
I am trying to compute nearest neighbour clustering on a Scipy sparse matrix returned from scikit-learn's DictVectorizer. However, when I try to compute the distance matrix with scikit-learn I get an
I am trying to create a large sparse matrix, 10^5 by 10^5 in R, but am running into memory issues. > Matrix(nrow=1e5,ncol=1e5,sparse=TRUE) Error in Matrix(nrow = 1e+05, ncol = 1e+05, sparse = TRUE)
Started learning octave recently. How do I generate a matrix from another matrix by applying a function to each element? eg: Apply 2x+1 or 2x/(x^2+1) or 1/x+3 to a 3x5 matrix A. The result should be
The type of matrix I am dealing with was created from a vector as shown below: Start with a 1-d vector V of length L. To create a matrix A from V with N rows, make the i'th column of A the first N ent
This Creating sparse matrix in MEX has a good example on mxCreateSparse. But this function return a double sparse matrix instead of single. If I want to return a single sparse matrix, what should I do
I have a very large and sparse matrix of size 180GB(text , 30k * 3M) containing only the entries and no additional data. I have to do matrix multiplication , inversion and some similar linear algebra
How to extract a block from a Eigen::SparseMatrix<double>. It seems there aren't the methods I used for the dense ones. ‘class Eigen::SparseMatrix<double>’ has no member named ‘topLeftCorn
I've created a matrix of values from a simulation that is stored in 20x7 matrix (20 observations across 7 columns of numbers; the matrix is called output). The columns are output from a simulation. Af
I have an N x N sparse matrix in Matlab, that has cell values indexed by (r,c) pairs such that r and c are unique id's. The problem is, that after converting this matrix into Python, all of the indic
Let label be a matrix of size N x 1 (type double) and data be a matrix of size N x M (type double). The entries in the Label matrix looks like [ 1; 23; 135; ....; 6] which conveys that the First row
I used clustal omega to get a distance matrix of 500 protein sequences (they are homologous to each other). I want to use affinity propagation to cluster these sequences. Initially, because I observed
There does not seem to be a method in scipy.sparse which gives the minimum of a sparse matrix. In particular, I seek the minimum of the columns. No method appears in the docs and numpy minimum does no
I have a very large (about 91 million non-zero entries) sparseMatrix() in R that looks like: > myMatrix a b c a . 1 2 b 1 . . c 2 . . I would like to convert it to a triangular matrix (upper or lo
Is it possible to create matrix of vectors in R? I mean the elements of this matrix must be vectors. For example mat[1,3] == c(6,8,9) i must create 40x40 matrix and i need to fill it manually.
I have Matrix and how get value from there? The data obtained from a file, so that the matrix can be of different sizes Thanks
I using the dot() function from numpy to multiply a matrix of 3x3 with a numpy.array of 1x3. The output is for example this: [[ 0.16666667 0.66666667 0.16666667]] which is of type: <class 'numpy.ma
I process rather large matrices in Python/Scipy. I need to extract rows from large matrix (which is loaded to coo_matrix) and use them as diagonal elements. Currently I do that in the following fashio
How can I create a sparse matrix from a list of dimension names? Suppose you have this matrix edgelist in a data frame: from to weight 1 4 a 1 2 5 b 2 3 6 c 3 It can be created like this: from <-
I have one matrix of mutation counts, say counts. This matrix has column names V1, V2,...,Vi,...Vn where not every i is there. Thus it can jump, such as V1, V2, V5 say. Further, most of columns ha
I know a matrix [x scale, y skew, x skew, y scale, trans x, trans y], and would like to get the angle in degrees. thanks!
I have an original matrix that looks something like this: (32, 69901) 1 (108, 69901) 1 (32, 69902) 1 (108, 69903) 1 (108, 69904) 1 (432, 69905) 1 (432, 69906) 1 (432, 69907) 1 I want to separate the
I've created a TermDocumentMatrix from the tm library in R. It looks something like this: > inspect(freq.terms) A document-term matrix (19 documents, 214 terms) Non-/sparse entries: 256/3810 Sparsi
When trying to directly set the data attribute of a sparse lil_matrix, I encounter very unexpected behavior. Can someone explain what is going on in the following simple example? My particular use ca
If you have a sparse matrix X: >> X = csr_matrix([[0,2,0,2],[0,2,0,1]]) >> print type(X) >> print X.todense() <class 'scipy.sparse.csr.csr_matrix'> [[0 2 0 2] [0 2 0 1]] And a
We can construct a sparse matrix from an index and value of non-zero element with the sparseMatrix or spMatrix. Is there any function convert a sparse matrix back to an index and value of all non-zero
I have a Table in MySQL with three columns: row-index, column-index and value, which I want to read in into a scipy csr_matrix. I use the Python-MySQL connector. There are 112,500 non-zero elements. T
I created a compressed sparse matrix, but while accessing to a positive index it complains that the index is negative: import scipy.sparse as sparse B= sparse.csc_matrix((110111213141516, 25)) B[11011
I have a matrix which is 36 x 2, but I want to seperate the elements to give me 18, 2 x 2 matrices from top to bottom. E.g. if I have a matrix: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 ... ... I want to spli
I'm currently working with sparse matrices, and I have to compare the computation time of sparse matrix-matrix multiplication with full matrix-matrix multiplication. The issue is that sparse matrix co
I'm thinking of using Boost's Sparse Matrix for a computation where minimal memory usage is the goal. Unfortunately, the documentation page didn't include a discussion of the sparse matrix implementat
Is there a built-in function in either slam package or Matrix package to convert a sparse matrix in simple triplet matrix form (from slam package) to a sparse matrix in dgTMatrix/dgCMatrix form (from
In MATLAB I have calculated the Fundamental matrix (of two images) using the normalized Eight point algorithm. From that I need to triangulate the corresponding image points in 3D space. From what I u
I have draw a sample u from random variable u~uniform(0,1) ; set.seed(123) num_samples <- 5 #number of samples num_time_periods <- 5 # number of years sample_u <- array(0,c(num_samples,num_ti
I'm using numpy and scipy. I have a large sparse matrix and I want to find the largest eigenvalue of the sparse matrix. How can I do that?
I have a scipy sparse matrix, where I need to add multiple rows (in blocks), say 1:30, then 45:50, etc. What is the most efficient way to do this?
I have a large dense matrix, say matrix A of size 10000 by 10000 and I need to extract a banded matrix of bandwidth say 10 from it, i.e., B(i,j) = A(i,j) if |i-j| <=10 B(i,j) = 0 otherwise What is
I have the following problem, I have a matrix generated by another program and written to a txt file, the file looks like this: ( I only included part of the matrix, total is 36 lines) 1 1 7.225655E+
I am trying create a data.frame from which to create a graph. I have a function and two vectors that I want to use as the two inputs. This is a bit simplified, but basically all I have is: relGPA <
I'm trying to calculate an expression of the form K = P*C.T*S^-1 (implementation of a Kalman filter) All the involved matrices are sparse, and I would of course like to avoid calculating the actual in
I have a scipy.sparse.csr matrix and would like to dump it to a CSV file. Is there a way to preserve the sparsity of the matrix and write it to a CSV?
I have a matrix A in CSC-format, of which I index just a single column b = A[:,col] resulting in a (n x 1) matrix. What I want to do is: v = M * b where M is a (n x n) matrix in CSR. The result v i
I got a problem when using octave sparse matrix. max(speye(65536)(:)) will result in a 0x0 variable. However, speye(65535) and speye(65537) works. How that happens? My octave version is 3.2.4 in Fedo