473,394 Members | 1,946 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,394 software developers and data experts.

Storage cost of a null column

What is the storage cost of a null entry in a column? i.e. does a null
entry of type integer, float8 or text consume the same amount of
storage as one that is filled? I ask because I have satellite data
which is transmitted via a dodgy RF link that drops data packets. This
means I have a number of columns in a table that are null. Moreover,
the operations people decided to use a compression scheme whereby
non-changing bit/integer values are not output at regular intervals
which also adds a considerable number of null entries into the columns.
Because of this, we made a decision that we would have hundreds of 2
column tables (timestamp, value) and use unions, intersections, and
joins to get what was needed. Unfortunately, this has made application
programming a real nightmare as we are often forced to reconstruct a
snapshot frame for the range of times either in C or have the app
create temporary tables in SQL and insert the relevant data prior to
selecting it. As it stands, we've ordered a new disk array and
provided that the storage costs are not that high, I will probably be
reorganising all this next week. If anyone has any other suggestions,
I'd be very keen to hear them.

Cheers,
Randall
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)

Nov 23 '05 #1
3 4621
Randall Skelton <sk*****@brutus.uwaterloo.ca> writes:
What is the storage cost of a null entry in a column?
If you have even one, all the rest in that row are free, so your scheme
sounds reasonable.

Null columns are simply not stored. There is a bitmap at the start of
the row with one bit per column, indicating which ones are null or not
null. However we omit the bitmap if all columns are nonnull in a
particular row. So it's reasonable to consider the cost of the first
null as being the size of the bitmap (N bits for an N-column table,
rounded up). The rest are free.
Because of this, we made a decision that we would have hundreds of 2
column tables (timestamp, value) and use unions, intersections, and


Narrow tables are a dead loss if you're concerned about storage space
--- you'll get eaten by the per-row overhead, which is a minimum of 28
bytes per row.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

Nov 23 '05 #2
A NULL costs almost nothing, in anycase, less than a real value.

There is the cost of the NULL bitmap, 1 bit per column for each column
if there are *any* NULLs in a row, but once you'ce got one, the rest
are free for that row.

This is in the documentation somewhere...

On Fri, Apr 02, 2004 at 12:18:47PM -0500, Randall Skelton wrote:
What is the storage cost of a null entry in a column? i.e. does a null
entry of type integer, float8 or text consume the same amount of
storage as one that is filled? I ask because I have satellite data
which is transmitted via a dodgy RF link that drops data packets. This
means I have a number of columns in a table that are null. Moreover,
the operations people decided to use a compression scheme whereby
non-changing bit/integer values are not output at regular intervals
which also adds a considerable number of null entries into the columns.
Because of this, we made a decision that we would have hundreds of 2
column tables (timestamp, value) and use unions, intersections, and
joins to get what was needed. Unfortunately, this has made application
programming a real nightmare as we are often forced to reconstruct a
snapshot frame for the range of times either in C or have the app
create temporary tables in SQL and insert the relevant data prior to
selecting it. As it stands, we've ordered a new disk array and
provided that the storage costs are not that high, I will probably be
reorganising all this next week. If anyone has any other suggestions,
I'd be very keen to hear them.

Cheers,
Randall


---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)
--
Martijn van Oosterhout <kl*****@svana.org> http://svana.org/kleptog/ This space intentionally left blank


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQFAbb9cY5Twig3Ge+YRAp8VAKCRR+QeYE9enrm0132bdp YRoXlKQQCbBfeu
fhLJtGh3Gd0wiva4eNEYM/E=
=5blo
-----END PGP SIGNATURE-----

Nov 23 '05 #3
Many thanks Tom and Martijn. It seems that I have a few tables to
reformat...
Narrow tables are a dead loss if you're concerned about storage space
--- you'll get eaten by the per-row overhead, which is a minimum of 28
bytes per row.

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Nov 23 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Leif K-Brooks | last post by:
I'm writing a relatively simple multi-user public Web application with Python. It's a rewrite of a similar application which used PHP+MySQL (not particularly clean code, either). My opinions on...
1
by: Jens H. Hamann | last post by:
Hi, I am having a strange problem with the storage parameters in Oracle 9.2.0.1 database. I create a tablespace with some default storage parameters but when i am adding a table to the tablespace...
3
by: Dan | last post by:
I am a relatively new user on Oracle 9.2.0.1 and I am having trouble performance tuning this production database. I am running a large query that joins two tables, document(3 mil) and entity(9...
2
by: LeTubs | last post by:
Hi I have few questions in realtion to arrays, I assume that they are not available as a data type, is this correct ? The reason why is that I want to store a large amount of data for a...
5
by: ujjc001 | last post by:
access subquery error: "not enough storage is available to complete this operation" Query--- SELECT TOP 100 PERCENT UPPER(dbo.Employee.last + N', ' + dbo.Employee.first) AS Employee,...
5
by: aneesh | last post by:
Hi all, I have a program, this works fine but if we declare static below "int i" it shows different storage class specifier. what will be the reason. #include <stdlib.h> static int i ; int...
3
by: Bas Wassink | last post by:
Hello there, I'm having trouble understanding a warning produced by 'splint', a code-checker. The warning produced is: keywords.c: (in function keyw_get_string) keywords.c:60:31: Released...
5
by: Brian P. Hammer | last post by:
I have data from multiple SQL tables, some of the tables will only have one row, while others multiple rows. I load a bunch of data from the various tables and add them to a third party grid. With...
7
by: StupidScript | last post by:
>From the manual "Storage Requirements": "ENUM('value1','value2',...) =1 or 2 bytes, depending on the number of enumeration values (65,535 values maximum)" This seems to mean: "a" = 1 byte...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.