What I have been working on

[ssl]

I have been working on a solution to extract data from a file format created and maintained by these guys (the link points to current version information, I have to deal with an older format):

Omnis Studio, Cross Platform, Web client, RAD tool, Rapid Application Development tool for Windows, Linux, Mac OS X, MySQL, PostgreSQL, Oracle, DB2, ODBC, JDBC, Sybase SQL database, Enterprise and Web applications

Let's just say it's not been a walk in the part, but a walk into many, many brick walls topped with barbed wires and broken glass (so it is difficult to climb over). But I've gotten to the point where I can extract strings (the data type/structure an individual can read, such as this post). I've to clean it up a little (make sure some of the stuff is not boring data (numbers referring to other records -- by extracting that data as well), and then it's to a spreadsheet (what the client wants)...

Be done with it, although I may consider (in the future, I'd rather not look at this code for years to come) offering data extraction services to clients; but not now (and no, I am not offering any services, so I am *not* spamming... *cough* donnie *ahem* ).

 

[RedRyder]

Spammer!




:jk

 

[ssl]

Spammer!




:jk

I can no help mee self! :willy_nilly::24:

 

[RedRyder]

I'm not big on computer programming..... but if you need help extracting a solution, I can maintain the links to provide such data necessary..... :ninja

 

[ssl]

I'm not big on computer programming..... but if you need help extracting a solution, I can maintain the links to provide such data necessary..... :ninja

I'll keep that in mind, as I am running the extraction (test run) to make sure I can get up to an increasing number (over successive test runs) of records.

After looking at the data I have extracted so far (from test run results) I may need to be ready to perform more extraction... (sigh)..

But I'll be done tonight, tomorrow morning (no sleep tonight!.. we, except for a little nap here or there..).

 

[ssl]

eh, 1k records max worked nicely (from last max of 75)..

I'm just worried about the ending parts (should not be, really).

*ups the ante - 10000 records max*

 

[HottyToddyChick]

*whoosh*

I can imagine that you have been a very stressed person though!

 

[ssl]

*whoosh*

I can imagine that you have been a very stressed person though!

I try to keep the stress levels down.

Namely, taking a nap, clears the mind.

----------------

Good thing is, I "unlocked" the max, so all records "recorded" are extractable.

Now to make them... fully readable. (any pointers to chase down and get that data).

 

[Codrus]

I try to keep the stress levels down.

Namely, taking a nap, clears the mind.

----------------

Good thing is, I "unlocked" the max, so all records "recorded" are extractable.

Now to make them... fully readable. (any pointers to chase down and get that data).

i just saw this (sorry to interupt ) but what exactly are you trying to do?

 

[Codrus]

from just a quick scan your looking to copy/paste multiple formats, like merging pdf with just about anything, am i right?

 

[ssl]

negative; i am attempting to convert a database (single file, thank god) to another, easily accessible and exported format.

proprietary format (omnis df1) to a spreadsheet (probably xls).

 

[Codrus]

how large is the data base?
is it compressed?
standard code?

 

[ssl]

how large is the data base?
is it compressed?
standard code?

Size: 58,610 KiB (~57.2 MiB)
Compressed: Not sure, but leaning towards no (so far, I am using flat binary)

Standard code?

 

[Codrus]

Size: 58,610 KiB (~57.2 MiB)
Compressed: Not sure, but leaning towards no (so far, I am using flat binary)

Standard code?

are you entering it manualy?

(I.d.10-T)....small joke :jk

 

[ssl]

are you entering it manualy?

(I.d.10-T)....small joke :jk

converting the database manually? fuck that.

writing a program, based on observations and a programming guide for the RDBM sys.

 

[Codrus]

converting the database manually? fuck that.

writing a program, based on observations and a programming guide for the RDBM sys.

ok then i take back the I.D.10-T

****takes off watch***

how much do you have?

 

[ssl]

not sure.

based off of observations, there appears to be 154695 records overall.
138502 are attributed to respective tables.

each tables' field definitions are partly parsed (6 of 8 bytes have purpose, 2 are type+subtype, 2 are size, 2 are startLocation)

currently, i have the field definitions driving the numerical-ish side of things, to allow me to extract the strings (strings are apparently defined with a maximum field length, or 0 (no max length, or unused, not known yet), but the actual data is <byte for length><n-bytes for string>);

i've nearly all the strings (there are more in the totalKnownRecords-countRecordsMatchedToTables number of records), I am about to add the field definitions to the output files (one per table, currently listing just the strings extracted).

While that is going on, I am also going to see if and where the relationships occur for the total-matched records, and figure out if i need a separate approach for parsing them, or if they are linked into the already known records.

then i have to reinsert the numerical data into the output files, as that will form a more complete picture (and these numbers may reference the extra records).

then i have to identify the field definitions, namely the first two bytes (type, subtype), so that i can begin the translation process.

once i am sure of the translation process, i'll let her rip, hopefully no errors, and see if the output results make sense; if they do, i am going to be done; otherwise... debugging.

:nod:

 

[Codrus]

all 8 have purpose some may be retro *( forgive me i have been drinking heavily on my day off)*

have you noticed any reoccurance in the 2 of 8?

 

[ssl]

all 8 have purpose some may be retro *( forgive me i have been drinking heavily on my day off)*

have you noticed any reoccurance in the 2 of 8?

yeaup.

0x0000

 

[Codrus]

ok, 0x000 may be a reference or security string either way that needs to be included or bypassed/looped as to not incur conflict when refernced/sampled.
i deal with in/out code on occasion and each in-code tags what is being used and what its being used on, i have a feeling you are dealing with something similar