Advertisement
Advertisement


In Java, what is the best way to determine the size of an object?


Question

I have an application that reads a CSV file with piles of data rows. I give the user a summary of the number of rows based on types of data, but I want to make sure that I don't read in too many rows of data and cause OutOfMemoryErrors. Each row translates into an object. Is there an easy way to find out the size of that object programmatically? Is there a reference that defines how large primitive types and object references are for a VM?

Right now, I have code that says read up to 32,000 rows, but I'd also like to have code that says read as many rows as possible until I've used 32MB of memory. Maybe that is a different question, but I'd still like to know.

2019/10/02
1
624
10/2/2019 6:50:19 PM

Accepted Answer

You can use the java.lang.instrument package

Compile and put this class in a JAR:

import java.lang.instrument.Instrumentation;

public class ObjectSizeFetcher {
    private static Instrumentation instrumentation;

    public static void premain(String args, Instrumentation inst) {
        instrumentation = inst;
    }

    public static long getObjectSize(Object o) {
        return instrumentation.getObjectSize(o);
    }
}

Add the following to your MANIFEST.MF:

Premain-Class: ObjectSizeFetcher

Use getObjectSize:

public class C {
    private int x;
    private int y;

    public static void main(String [] args) {
        System.out.println(ObjectSizeFetcher.getObjectSize(new C()));
    }
}

Invoke with:

java -javaagent:ObjectSizeFetcherAgent.jar C
2016/06/28
468
6/28/2016 6:24:59 PM

You should use jol, a tool developed as part of the OpenJDK project.

JOL (Java Object Layout) is the tiny toolbox to analyze object layout schemes in JVMs. These tools are using Unsafe, JVMTI, and Serviceability Agent (SA) heavily to decoder the actual object layout, footprint, and references. This makes JOL much more accurate than other tools relying on heap dumps, specification assumptions, etc.

To get the sizes of primitives, references and array elements, use VMSupport.vmDetails(). On Oracle JDK 1.8.0_40 running on 64-bit Windows (used for all following examples), this method returns

Running 64-bit HotSpot VM.
Using compressed oop with 0-bit shift.
Using compressed klass with 3-bit shift.
Objects are 8 bytes aligned.
Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes]
Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes]

You can get the shallow size of an object instance using ClassLayout.parseClass(Foo.class).toPrintable() (optionally passing an instance to toPrintable). This is only the space consumed by a single instance of that class; it does not include any other objects referenced by that class. It does include VM overhead for the object header, field alignment and padding. For java.util.regex.Pattern:

java.util.regex.Pattern object internals:
 OFFSET  SIZE        TYPE DESCRIPTION                    VALUE
      0     4             (object header)                01 00 00 00 (0000 0001 0000 0000 0000 0000 0000 0000)
      4     4             (object header)                00 00 00 00 (0000 0000 0000 0000 0000 0000 0000 0000)
      8     4             (object header)                cb cf 00 20 (1100 1011 1100 1111 0000 0000 0010 0000)
     12     4         int Pattern.flags                  0
     16     4         int Pattern.capturingGroupCount    1
     20     4         int Pattern.localCount             0
     24     4         int Pattern.cursor                 48
     28     4         int Pattern.patternLength          0
     32     1     boolean Pattern.compiled               true
     33     1     boolean Pattern.hasSupplementary       false
     34     2             (alignment/padding gap)        N/A
     36     4      String Pattern.pattern                (object)
     40     4      String Pattern.normalizedPattern      (object)
     44     4        Node Pattern.root                   (object)
     48     4        Node Pattern.matchRoot              (object)
     52     4       int[] Pattern.buffer                 null
     56     4         Map Pattern.namedGroups            null
     60     4 GroupHead[] Pattern.groupNodes             null
     64     4       int[] Pattern.temp                   null
     68     4             (loss due to the next object alignment)
Instance size: 72 bytes (reported by Instrumentation API)
Space losses: 2 bytes internal + 4 bytes external = 6 bytes total

You can get a summary view of the deep size of an object instance using GraphLayout.parseInstance(obj).toFootprint(). Of course, some objects in the footprint might be shared (also referenced from other objects), so it is an overapproximation of the space that could be reclaimed when that object is garbage collected. For the result of Pattern.compile("^[a-zA-Z0-9_.+-][email protected][a-zA-Z0-9-]+\\.[a-zA-Z0-9-.]+$") (taken from this answer), jol reports a total footprint of 1840 bytes, of which only 72 are the Pattern instance itself.

java.util.regex.Pattern instance footprint:
     COUNT       AVG       SUM   DESCRIPTION
         1       112       112   [C
         3       272       816   [Z
         1        24        24   java.lang.String
         1        72        72   java.util.regex.Pattern
         9        24       216   java.util.regex.Pattern$1
        13        24       312   java.util.regex.Pattern$5
         1        16        16   java.util.regex.Pattern$Begin
         3        24        72   java.util.regex.Pattern$BitClass
         3        32        96   java.util.regex.Pattern$Curly
         1        24        24   java.util.regex.Pattern$Dollar
         1        16        16   java.util.regex.Pattern$LastNode
         1        16        16   java.util.regex.Pattern$Node
         2        24        48   java.util.regex.Pattern$Single
        40                1840   (total)

If you instead use GraphLayout.parseInstance(obj).toPrintable(), jol will tell you the address, size, type, value and path of field dereferences to each referenced object, though that's usually too much detail to be useful. For the ongoing pattern example, you might get the following. (Addresses will likely change between runs.)

java.util.regex.Pattern object externals:
          ADDRESS       SIZE TYPE                             PATH                           VALUE
         d5e5f290         16 java.util.regex.Pattern$Node     .root.next.atom.next           (object)
         d5e5f2a0        120 (something else)                 (somewhere else)               (something else)
         d5e5f318         16 java.util.regex.Pattern$LastNode .root.next.next.next.next.next.next.next (object)
         d5e5f328      21664 (something else)                 (somewhere else)               (something else)
         d5e647c8         24 java.lang.String                 .pattern                       (object)
         d5e647e0        112 [C                               .pattern.value                 [^, [, a, -, z, A, -, Z, 0, -, 9, _, ., +, -, ], +, @, [, a, -, z, A, -, Z, 0, -, 9, -, ], +, \, ., [, a, -, z, A, -, Z, 0, -, 9, -, ., ], +, $]
         d5e64850        448 (something else)                 (somewhere else)               (something else)
         d5e64a10         72 java.util.regex.Pattern                                         (object)
         d5e64a58        416 (something else)                 (somewhere else)               (something else)
         d5e64bf8         16 java.util.regex.Pattern$Begin    .root                          (object)
         d5e64c08         24 java.util.regex.Pattern$BitClass .root.next.atom.val$rhs        (object)
         d5e64c20        272 [Z                               .root.next.atom.val$rhs.bits   [false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, true, false, true, true, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, true, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false]
         d5e64d30         24 java.util.regex.Pattern$1        .root.next.atom.val$lhs.val$lhs.val$lhs.val$lhs.val$lhs.val$lhs (object)
         d5e64d48         24 java.util.regex.Pattern$1        .root.next.atom.val$lhs.val$lhs.val$lhs.val$lhs.val$lhs.val$rhs (object)
         d5e64d60         24 java.util.regex.Pattern$5        .root.next.atom.val$lhs.val$lhs.val$lhs.val$lhs.val$lhs (object)
         d5e64d78         24 java.util.regex.Pattern$1        .root.next.atom.val$lhs.val$lhs.val$lhs.val$lhs.val$rhs (object)
         d5e64d90         24 java.util.regex.Pattern$5        .root.next.atom.val$lhs.val$lhs.val$lhs.val$lhs (object)
         d5e64da8         24 java.util.regex.Pattern$5        .root.next.atom.val$lhs.val$lhs.val$lhs (object)
         d5e64dc0         24 java.util.regex.Pattern$5        .root.next.atom.val$lhs.val$lhs (object)
         d5e64dd8         24 java.util.regex.Pattern$5        .root.next.atom.val$lhs        (object)
         d5e64df0         24 java.util.regex.Pattern$5        .root.next.atom                (object)
         d5e64e08         32 java.util.regex.Pattern$Curly    .root.next                     (object)
         d5e64e28         24 java.util.regex.Pattern$Single   .root.next.next                (object)
         d5e64e40         24 java.util.regex.Pattern$BitClass .root.next.next.next.atom.val$rhs (object)
         d5e64e58        272 [Z                               .root.next.next.next.atom.val$rhs.bits [false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, true, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false]
         d5e64f68         24 java.util.regex.Pattern$1        .root.next.next.next.atom.val$lhs.val$lhs.val$lhs (object)
         d5e64f80         24 java.util.regex.Pattern$1        .root.next.next.next.atom.val$lhs.val$lhs.val$rhs (object)
         d5e64f98         24 java.util.regex.Pattern$5        .root.next.next.next.atom.val$lhs.val$lhs (object)
         d5e64fb0         24 java.util.regex.Pattern$1        .root.next.next.next.atom.val$lhs.val$rhs (object)
         d5e64fc8         24 java.util.regex.Pattern$5        .root.next.next.next.atom.val$lhs (object)
         d5e64fe0         24 java.util.regex.Pattern$5        .root.next.next.next.atom      (object)
         d5e64ff8         32 java.util.regex.Pattern$Curly    .root.next.next.next           (object)
         d5e65018         24 java.util.regex.Pattern$Single   .root.next.next.next.next      (object)
         d5e65030         24 java.util.regex.Pattern$BitClass .root.next.next.next.next.next.atom.val$rhs (object)
         d5e65048        272 [Z                               .root.next.next.next.next.next.atom.val$rhs.bits [false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, true, true, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false]
         d5e65158         24 java.util.regex.Pattern$1        .root.next.next.next.next.next.atom.val$lhs.val$lhs.val$lhs.val$lhs (object)
         d5e65170         24 java.util.regex.Pattern$1        .root.next.next.next.next.next.atom.val$lhs.val$lhs.val$lhs.val$rhs (object)
         d5e65188         24 java.util.regex.Pattern$5        .root.next.next.next.next.next.atom.val$lhs.val$lhs.val$lhs (object)
         d5e651a0         24 java.util.regex.Pattern$1        .root.next.next.next.next.next.atom.val$lhs.val$lhs.val$rhs (object)
         d5e651b8         24 java.util.regex.Pattern$5        .root.next.next.next.next.next.atom.val$lhs.val$lhs (object)
         d5e651d0         24 java.util.regex.Pattern$5        .root.next.next.next.next.next.atom.val$lhs (object)
         d5e651e8         24 java.util.regex.Pattern$5        .root.next.next.next.next.next.atom (object)
         d5e65200         32 java.util.regex.Pattern$Curly    .root.next.next.next.next.next (object)
         d5e65220        120 (something else)                 (somewhere else)               (something else)
         d5e65298         24 java.util.regex.Pattern$Dollar   .root.next.next.next.next.next.next (object)

The "(something else)" entries describe other objects in the heap that are not part of this object graph.

The best jol documentation is the jol samples in the jol repository. The samples demonstrate common jol operations and show how you can use jol to analyze VM and garbage collector internals.

2015/05/04

I accidentally found a java class "jdk.nashorn.internal.ir.debug.ObjectSizeCalculator", already in jdk, which is easy to use and seems quite useful for determining the size of an object.

System.out.println(ObjectSizeCalculator.getObjectSize(new gnu.trove.map.hash.TObjectIntHashMap<String>(12000, 0.6f, -1)));
System.out.println(ObjectSizeCalculator.getObjectSize(new HashMap<String, Integer>(100000)));
System.out.println(ObjectSizeCalculator.getObjectSize(3));
System.out.println(ObjectSizeCalculator.getObjectSize(new int[]{1, 2, 3, 4, 5, 6, 7 }));
System.out.println(ObjectSizeCalculator.getObjectSize(new int[100]));

results:

164192
48
16
48
416
2016/09/09

Some years back Javaworld had an article on determining the size of composite and potentially nested Java objects, they basically walk through creating a sizeof() implementation in Java. The approach basically builds on other work where people experimentally identified the size of primitives and typical Java objects and then apply that knowledge to a method that recursively walks an object graph to tally the total size.

It is always going to be somewhat less accurate than a native C implementation simply because of the things going on behind the scenes of a class but it should be a good indicator.

Alternatively a SourceForge project appropriately called sizeof that offers a Java5 library with a sizeof() implementation.

P.S. Do not use the serialization approach, there is no correlation between the size of a serialized object and the amount of memory it consumes when live.

2020/07/12

Firstly "the size of an object" isn't a well-defined concept in Java. You could mean the object itself, with just its members, the Object and all objects it refers to (the reference graph). You could mean the size in memory or the size on disk. And the JVM is allowed to optimise things like Strings.

So the only correct way is to ask the JVM, with a good profiler (I use YourKit), which probably isn't what you want.

However, from the description above it sounds like each row will be self-contained, and not have a big dependency tree, so the serialization method will probably be a good approximation on most JVMs. The easiest way to do this is as follows:

 Serializable ser;
 ByteArrayOutputStream baos = new ByteArrayOutputStream();
 ObjectOutputStream oos = new ObjectOutputStream(baos);
 oos.writeObject(ser);
 oos.close();
 return baos.size();

Remember that if you have objects with common references this will not give the correct result, and size of serialization will not always match size in memory, but it is a good approximation. The code will be a bit more efficient if you initialise the ByteArrayOutputStream size to a sensible value.

2018/02/03

If you would just like to know how much memory is being used in your JVM, and how much is free, you could try something like this:

// Get current size of heap in bytes
long heapSize = Runtime.getRuntime().totalMemory();

// Get maximum size of heap in bytes. The heap cannot grow beyond this size.
// Any attempt will result in an OutOfMemoryException.
long heapMaxSize = Runtime.getRuntime().maxMemory();

// Get amount of free memory within the heap in bytes. This size will increase
// after garbage collection and decrease as new objects are created.
long heapFreeSize = Runtime.getRuntime().freeMemory();

edit: I thought this might be helpful as the question author also stated he would like to have logic that handles "read as many rows as possible until I've used 32MB of memory."

2014/07/16

Source: https://stackoverflow.com/questions/52353
Licensed under: CC-BY-SA with attribution
Not affiliated with: Stack Overflow
Email: [email protected]