8 bit data expands

Jun 25, 2010 at 8:06 AM
Hi, I'm trying to compress 8 bit data. At the end of compression, the compressed data is larger than the original data. In what conditions will charls expand data ? I need help figuring this out. Thanks very much. Ashwin
Jun 26, 2010 at 8:53 PM

That's hard to guess. You're the first to report this.

Could you post/mail me an example image for which you see this happening? And, summarize the compression parameters you select?

Jun 27, 2010 at 7:01 AM
That was a bug in my code. Sorry about that. While creating a byte buffer in JNI, I assumed the values would be unsigned. This means that data did expand when the values were between -127 to 128 instead of 0 to 255. I have another question about the decoding api. the method: CHARLS_IMEXPORT JLS_ERROR JpegLsDecode(void* pdataUncompressed, size_t cbyteUncompressed, const void* pdataCompressed, size_t cbyteCompressed, JlsParamaters* info = NULL) does not give the number of bytes written to Uncompressed array. I usually create a safely large array but I do not know exactly how many bytes to read from it. In Java it might be safe assume that the rest of the bytes are null but it might not be the case C. In any case, it would be nice if the method told you the size of the output data. The Encode API gives this information. I'd be very grateful if this could be done. thanks, Ashwin
Jun 30, 2010 at 1:02 AM
Edited Jun 30, 2010 at 1:08 AM

hey  jdv,  

really need your help here :  
The whole signed/unsigned issue is really confusing. So I thought I'd copy all data from the java array to an unsigned char array in c++.   


Following is my code :

        unsigned long i, lenOUT, lenIN;
 printf("this works! 1\n");
 jshort *uncompressedData = env->GetShortArrayElements(inputArray, 0);
 lenIN = env->GetArrayLength(inputArray);
 lenOUT = (size_t)cByteUncompressed;
 printf("this works! 2\n");
 printf("lenIN: %d\n", lenIN);
 printf("cByteUncompressed: %d\n", cByteUncompressed);
 unsigned char *pdataCompressed = (unsigned char *)malloc((size_t)lenOUT * sizeof(unsigned char)); // creating new arrays for input and output
 unsigned char *newUncompressedData = (unsigned char *)malloc((size_t)lenIN * sizeof(unsigned char));
 printf("this works! 3\n");

 for(i=0; i<lenIN; i++) {
  unsigned char val = (unsigned char)(uncompressedData[i] & 0xFF);
  newUncompressedData[i] = val;   // copying 8 bit data. Original data is 8 bit data stored in 16 bit short.

 env->ReleaseShortArrayElements(inputArray, uncompressedData, 0);

 printf("this works! 4\n");

 JlsParamaters params = JlsParamaters();
 jfieldID fid;
 jclass jlsParams;

 jlsParams = env->GetObjectClass(pparams);
 fid = env->GetFieldID(jlsParams, "height", "I");
 params.height = env->GetIntField(pparams, fid);

 fid = env->GetFieldID(jlsParams, "width", "I");
 params.width = env->GetIntField(pparams, fid);

 fid = env->GetFieldID(jlsParams, "bitspersample", "I");
 params.bitspersample = env->GetIntField(pparams, fid);

 fid = env->GetFieldID(jlsParams, "bytesperline", "I");
 params.bytesperline = env->GetIntField(pparams, fid);

 fid = env->GetFieldID(jlsParams, "components", "I");
 params.components = env->GetIntField(pparams, fid);

 fid = env->GetFieldID(jlsParams, "allowedlossyerror", "I");
 params.allowedlossyerror = env->GetIntField(pparams, fid);

 jclass exc = env->FindClass("java/lang/IllegalArgumentException");
 if(params.components == 1){
  params.ilv = ILV_NONE;
 }else if (params.components == 4){
  params.ilv = ILV_LINE;
  env->ThrowNew(exc, "incorrect value encountered for JlsParameters.components");
 }else if (params.components == 3){
  params.ilv = ILV_LINE;
  params.colorTransform = 1;
  env->ThrowNew(exc, "incorrect value encountered for JlsParameters.components");
  if(exc == NULL){
   return (jint)1;
   env->ThrowNew(exc, "incorrect value encountered for JlsParameters.components");
 size_t pcByteWritten;

 printf("this works! 5\n");
 error = JpegLsEncode(pdataCompressed, (size_t)lenOUT, &pcByteWritten, newUncompressedData, (size_t)lenIN, &params);
 printf("this works! 6\n");

The output is as follows :

this works! 1
this works! 2
lenIN: 421201
cByteUncompressed: 421201
this works! 3
this works! 4
this works! 5
# A fatal error has been detected by the Java Runtime Envi
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x1001ecd
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) Client VM (16.3-b01 mixed mode
# Problematic frame:
# C  [charls.dll+0x1ecd3]
# An error report file with more information is saved as:
# C:\workspace\JAI\bin\hs_err_pid2644.log
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in n
# See problematic frame for where to report the bug.

[error occurred during error reporting , id 0xc0000005]


JlsParameters are as follows :

<font size="2">


</font>width = col;

 params.height = row;

<font size="2">


</font>bitspersample = 8;<font size="2"> </font><font size="2">


</font>bytesperline = (params.bitspersample)*params.width / 8;
<font size="2"> </font>
<font size="2">


</font>components = 1;
<font size="2"> </font>
<font size="2">


</font>allowedlossyerror = 0;

It crashes right at the method call. If I just use jshort array for input and output, things work fine. But jshort is signed so I don't want to use that. I don't understand why is there an issue with unsigned char

please suggest ..


Jun 30, 2010 at 2:09 AM

I increased the size of pdataCompressed ( output buffer ) and it worked. Which means if the output buffer is too small, the program just crashes.

Anyway, this confirms that data expands. Can you give me your email address so I can email you the data. I split the image in parts and encode each part separately.

So I will give you a binary file with bytes. You can just construct a x*y array and try to conpress it. It expands!