Be Excellent To Each Other

Hi Guys,

I hope this might be quick and painless, here my problem:

I have a dataset of over 10,000 rows each with around 100 columns. I can read this data (and it's only required to be read) from a Database and place it in a dataset very quickly (less than a fraction of a second) but when I iterate through the dataset to populate arrays and other program guff it takes ages (7 Seconds or so). I've checked what I'm populating from the dataset and that in itself is super quick and adds little overhead, all I'm doing is populating another data structure - little in the way of calculations apart from conversions.

What I don't understand is why it's taking so long to iterate through the dataset, when it's taken hardly any time to fill it.

* Are datasets the best way in C++ to manage database held data ( I can't think of anything else)?
* Are there any performance settings that might improve dataset read speeds?

This is all theory, as I'm not actually doing the work myself but I seem to be at the end of the sharp pointy stick named "get shit done".

All using C++ of which I think is .NET.

Cheers for any pointers (not &actual *pointers) or ideas

Does the original dataset fit into memory but the new one doesn't?

Any chance of seeing the code?

Well there's only one dataset. It's populated from the DB. It's then used to populate a different data structure.

I don't have the code to show, I've only been able to review quickly over a webex. If I can get it and allowed, I'll post it.

Did find this... although C# I assume all .NET is equal: http://www.eggheadcafe.com/articles/20030205.asp

Unless there's something hideous in the underlying C++ code, I think you're reaching the performance limit of the DataSet. Here's an article comparing the performance of the DataSet to the DataReader: http://www.devx.com/vb2themax/Article/1 ... 54?pf=true

Personally I don't use DataSets, but more because they promote tight coupling and limit flexibility, although I suspect they're perfectly good for smallish apps.

Cheers guys. I'll see what else I can dig up.