473,327 Members | 1,920 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,327 software developers and data experts.

Reading Binary Image Data

Hey Guys.
I'm looking for some info on how to read binary image data. Basicly RAW image files are strings of bits with say 2 bytes per pixel. What i need to do is read the value of each pixel in turn and store then in another variable, px1, pix2 etc. I'm running an iterative algorithm on 2 pixels at a time then moving to the next two pixels.
I am having alot of problems however reading the data. from the net and various books i see the readline method however this looks for new line charicters. I need something that will read the value of a certain number of bits from the file.

for example to read the first pixel data which is 16bits long i need something that will just read those 16bits and return its value.
Thanks.
Ed
Jun 26 '07 #1
11 17577
bvdet
2,851 Expert Mod 2GB
This will read two bytes at a time:
Expand|Select|Wrap|Line Numbers
  1. >>> print [f.read(2) for _ in range(12)]
  2. ['96', '80', '07', '91', '37', '84', '41', '71', '15', '25', '18', '99']
  3. >>> 
This was done on a text file. You should look at PIL link
Jun 26 '07 #2
bartonc
6,596 Expert 4TB
This will read two bytes at a time:
Expand|Select|Wrap|Line Numbers
  1. >>> print [f.read(2) for _ in range(12)]
  2. ['96', '80', '07', '91', '37', '84', '41', '71', '15', '25', '18', '99']
  3. >>> 
This was done on a text file. You should look at PIL link
I imagine that you opened that text file in binary mode...
Expand|Select|Wrap|Line Numbers
  1. f = open('somefile.txt', 'b')
first.
Jun 26 '07 #3
bartonc
6,596 Expert 4TB
Hey Guys.
I'm looking for some info on how to read binary image data. Basicly RAW image files are strings of bits with say 2 bytes per pixel. What i need to do is read the value of each pixel in turn and store then in another variable, px1, pix2 etc. I'm running an iterative algorithm on 2 pixels at a time then moving to the next two pixels.
I am having alot of problems however reading the data. from the net and various books i see the readline method however this looks for new line charicters. I need something that will read the value of a certain number of bits from the file.

for example to read the first pixel data which is 16bits long i need something that will just read those 16bits and return its value.
Thanks.
Ed
I'm guessing that you know the inner workings of the image format that you are working on and can separate the format information (header) from the actual pixels of the image (data).

For working on large data sets, SciPy arrays have some features that native python lists lack.
Jun 26 '07 #4
Thanks guys.
Will investigate those options once i have time.
The RAW format i am working with is literally just raw data, it doesnt have a header file or start and stop bytes. The pixels have a bitdepth of 2bytes however the actual image data is only 10bits out of those 16, the rest are parity and padding bits.

Edit:
Ok just looking at the f.read(2) code above. The method returns the value of those two bytes as a string. The returned value is a hex value but is a string representation. How do i convert this string value into the actual hex value.
For example:

>>> f = open(r'C:\EFisher_ST_project07\RAW test images\724_day_mac_AV_on.raw', 'rb')
>>> pixel = f.read(2)
>>> pixel
'\xff\x00'
>>> type(pixel)
<type 'str'>

Cheers
Ed
Jun 26 '07 #5
bvdet
2,851 Expert Mod 2GB
Thanks guys.
Will investigate those options once i have time.
The RAW format i am working with is literally just raw data, it doesnt have a header file or start and stop bytes. The pixels have a bitdepth of 2bytes however the actual image data is only 10bits out of those 16, the rest are parity and padding bits.

Edit:
Ok just looking at the f.read(2) code above. The method returns the value of those two bytes as a string. The returned value is a hex value but is a string representation. How do i convert this string value into the actual hex value.
For example:

>>> f = open(r'C:\EFisher_ST_project07\RAW test images\724_day_mac_AV_on.raw', 'rb')
>>> pixel = f.read(2)
>>> pixel
'\xff\x00'
>>> type(pixel)
<type 'str'>

Cheers
Ed
Maybe this will help:
Expand|Select|Wrap|Line Numbers
  1. import re
  2. patt = re.compile(r'(?<=\\x)[a-f0-9]+')
  3. s = repr('\xff\x00')
  4.  
  5. for hexNum in patt.findall(s):
  6.     print int(hexNum, 16)
Output:
>>> 255
0
>>>
>>> patt.findall(s)
['ff', '00']
>>>
Jun 26 '07 #6
bartonc
6,596 Expert 4TB
Thanks guys.
Will investigate those options once i have time.
The RAW format i am working with is literally just raw data, it doesnt have a header file or start and stop bytes. The pixels have a bitdepth of 2bytes however the actual image data is only 10bits out of those 16, the rest are parity and padding bits.

Edit:
Ok just looking at the f.read(2) code above. The method returns the value of those two bytes as a string. The returned value is a hex value but is a string representation. How do i convert this string value into the actual hex value.
For example:

>>> f = open(r'C:\EFisher_ST_project07\RAW test images\724_day_mac_AV_on.raw', 'rb')
>>> pixel = f.read(2)
>>> pixel
'\xff\x00'
>>> type(pixel)
<type 'str'>

Cheers
Ed
I've never played with the struct module before, but here's what I found:
>>> import struct
>>> a = '\xff\xff'
>>> b = struct.unpack('BB', a)
>>> b
(255, 255)
>>> b = struct.unpack('H', a)
>>> b
(65535,)
>>>
Jun 27 '07 #7
bartonc
6,596 Expert 4TB
I've never played with the struct module before, but here's what I found:
>>> import struct
>>> a = '\xff\xff'
>>> b = struct.unpack('BB', a)
>>> b
(255, 255)
>>> b = struct.unpack('H', a)
>>> b
(65535,)
>>>
How about this:

>>> a = a * 8
>>> a
'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\ xff\xff\xff\xff'
>>> import array
>>> b = array.array('H',a)
>>> b
array('H', [65535, 65535, 65535, 65535, 65535, 65535, 65535, 65535])
>>>
Jun 27 '07 #8
bartonc
6,596 Expert 4TB
How about this:

>>> a = a * 8
>>> a
'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\ xff\xff\xff\xff'
>>> import array
>>> b = array.array('H',a)
>>> b
array('H', [65535, 65535, 65535, 65535, 65535, 65535, 65535, 65535])
>>>
Or better yet, use the array module to read the data directly:
fromfile( f, n)

Read n items (as machine values) from the file object f and append them to the end of the array. If less than n items are available, EOFError is raised, but the items that were available are still inserted into the array. f must be a real built-in file object; something else with a read() method won't do.
Jun 27 '07 #9
Cheers guys, will give these ideas a try and see how they turn out.
Thanks.
Ed
Jun 27 '07 #10
Hey Guys. I managed to do it, thats for your help.
Here is what i came up with:
Expand|Select|Wrap|Line Numbers
  1. def GetPxValue(openfile, bitdepth): #can iterate this part to get the next pixel
  2.     read_bytes = operator.div(bitdepth, 8)  
  3.     px_value = openfile.read(read_bytes)    
  4.     if (bitdepth == 8):
  5.         unpack_code = 'B'
  6.     if (bitdepth == 16):
  7.         unpack_code = 'BB'          
  8.     if (bitdepth == 32):
  9.         unpack_code = 'BBBB'
  10.     hex_value = struct.unpack(unpack_code, px_value)  
  11.     msbyte = Dec2Bin(hex_value[0])
  12.     if (bitdepth == 16):
  13.         lsbyte = Dec2Bin(hex_value[1])      
  14.         for i in range(8):          
  15.             msbyte.append(lsbyte[i])
  16.     if (bitdepth == 32):
  17.         byte2 = Dec2Bin(hex_value[1])
  18.         byte3 = Dec2Bin(hex_value[2])       
  19.         lsbyte = Dec2Bin(hex_value[3])
  20.         for i in range(8):
  21.             msbyte.append(byte2[i])
  22.         for i in range(8):
  23.             msbyte.append(byte3[i])
  24.         for i in range(8):
  25.             msbyte.append(lsbyte[i]) 
  26.     return msbyte 
  27. #return the whole bitdepth long pixel value as a list ready for filtering
  28.  
Note: Dec2Bin is a decimal to binary converter that returns a binary value as a list. the list is necessary for filtering in the next stage.
Cheers.
Ed
Jun 27 '07 #11
bartonc
6,596 Expert 4TB
Hey Guys. I managed to do it, thats for your help.
Here is what i came up with:
Expand|Select|Wrap|Line Numbers
  1. def GetPxValue(openfile, bitdepth): #can iterate this part to get the next pixel
  2.     read_bytes = operator.div(bitdepth, 8)  
  3.     px_value = openfile.read(read_bytes)    
  4.     if (bitdepth == 8):
  5.         unpack_code = 'B'
  6.     if (bitdepth == 16):
  7.         unpack_code = 'BB'          
  8.     if (bitdepth == 32):
  9.         unpack_code = 'BBBB'
  10.     hex_value = struct.unpack(unpack_code, px_value)  
  11.     msbyte = Dec2Bin(hex_value[0])
  12.     if (bitdepth == 16):
  13.         lsbyte = Dec2Bin(hex_value[1])      
  14.         for i in range(8):          
  15.             msbyte.append(lsbyte[i])
  16.     if (bitdepth == 32):
  17.         byte2 = Dec2Bin(hex_value[1])
  18.         byte3 = Dec2Bin(hex_value[2])       
  19.         lsbyte = Dec2Bin(hex_value[3])
  20.         for i in range(8):
  21.             msbyte.append(byte2[i])
  22.         for i in range(8):
  23.             msbyte.append(byte3[i])
  24.         for i in range(8):
  25.             msbyte.append(lsbyte[i]) 
  26.     return msbyte 
  27. #return the whole bitdepth long pixel value as a list ready for filtering
  28.  
Note: Dec2Bin is a decimal to binary converter that returns a binary value as a list. the list is necessary for filtering in the next stage.
Cheers.
Ed
Hi Ed. It's great when members post the solutions that they have come up with. I've added CODE tags to your post. Instructions on how to do this are on the right hand side of the page while you are posting or replying.

I felt sure that you would go with the array thing when I discovered it. I think that you will get a performance boost be reading larger chunks and working on slices of the array. The cool thing about python is that you may use one approach to get things working, then, with a little tweaking, make it work really well.

Have fun,
Barton
Jun 27 '07 #12

Sign in to post your reply or Sign up for a free account.

Similar topics

7
by: John | last post by:
I have over 5000 thumbnail pictures of size 5kb each. I would like to able to load all 5000 pictures and view 50 per page using mysql_data_seek(). I would like to know what are the advantages and...
2
by: Albert Tu | last post by:
Hi, I am learning and pretty new to Python and I hope your guys can give me a quick start. I have an about 1G-byte binary file from a flat panel x-ray detector; I know at the beggining there...
2
by: jimmyfishbean | last post by:
Hi, I am using VB6, SAX (implementing IVBSAXContentHandler). I need to extract binary encoded data (images) from large XML files and decode this data and generate the appropriate images onto...
3
by: dale zhang | last post by:
Hi, I am trying to read an image from MS Access DB based on the following article: http://www.vbdotnetheaven.com/Code/Sept2003/2175.asp The article author is using PictureBox for windows...
4
by: dale zhang | last post by:
Hi, I am trying to save and read an image from MS Access DB based on the following article: http://www.vbdotnetheaven.com/Code/Sept2003/2175.asp Right now, I saved images without any...
2
by: Denise Smith | last post by:
Hello, I'm wondering if anyone can help me out here? I want to be able to browse records in a database where one of the fields contains an image. I think I might have to extract the image...
2
by: Chucker | last post by:
Hi Community, I think I can store Binary Data in SQL Server but when I try to retrieve it, I always only get one byte. I think I stored my Binary Data in SQL Server in a Colum of Type Image....
2
by: Ed | last post by:
Hope someone can help me out... I have been tasked to read some image data from an sql database and save the files to flat files. OK, sounds easy as I'v used BLOBs before. But this is an old...
6
by: jcasique.torres | last post by:
Hi everyboy. I trying to create a C promang in an AIX System to read JPG files but when it read just the first 4 bytes when it found a DLE character (^P) doesn't read anymore. I using fread...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.