Feed-forward Gaussian splatting models have demonstrated remarkable effectiveness in reconstructing threedimensional (3D) objects from a few two-dimensional (2D) images, even if they are unseen. As existing methods typically predict Gaussian primitives uniformly across the 3D space, most primitives are placed in non-object regions. This may hinder the representation of fine object details. This paper proposes a Voxel- Selective Gaussian Splatting model (VS-Splat), a new end-to-end feed-forward Gaussian splatting framework that predicts many primitives only within selected voxels that are likely to belong to an object, without 3D structural supervision. To achieve this, we propose a new learnable voxel selection approach that identifies object-centric voxels only with 2D rendering supervision. Our sparse-view rendering experiments with three benchmark datasets show that proposed VS-Splat outperforms several stateof- the-art methods. We further demonstrate its effectiveness as a backbone for an existing densification method and show that an optional extension improves its robustness to inaccurate camera pose estimates.