www.ice-graphics.com Forum Index www.ice-graphics.com
The main forum for the ICE-Graphics software
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Is ICE-EEC Par3 + Please improve interface
Goto page 1, 2  Next
 
Post new topic   Reply to topic    www.ice-graphics.com Forum Index -> Suggestions
View previous topic :: View next topic  
Author Message
Z3r0



Joined: 12 Jul 2005
Posts: 8

PostPosted: Tue Jul 12, 2005 9:29 pm    Post subject: Is ICE-EEC Par3 + Please improve interface Reply with quote

Is ICE-EEC Par3?

Also: can you add a feature that lets you select the number of files that can be recovered like quickpar and improve the interface a little as it's a bit hard to understand tx

The header redundancy stuff is of no meaning or importance to me, can you just get rid of it or put it under advanced settings or something or maybe add some mouseover hints.
Back to top
View user's profile Send private message
ICE Graphics
Site Admin


Joined: 31 Mar 2003
Posts: 430

PostPosted: Wed Jul 13, 2005 8:03 am    Post subject: Re: Is ICE-EEC Par3 + Please improve interface Reply with quote

Z3r0 wrote:
Is ICE-EEC Par3?

That it is more important for you: functionality or project name?

Z3r0 wrote:
Also: can you add a feature that lets you select the number of files that can be recovered like quickpar

What mean "number of files"? Are you mean partial recovery?

Z3r0 wrote:
and improve the interface a little as it's a bit hard to understand tx

Try to explain, what do you want to change.

Z3r0 wrote:
The header redundancy stuff is of no meaning or importance to me, can you just get rid of it or put it under advanced settings or something or maybe add some mouseover hints.

OK
Back to top
View user's profile Send private message Visit poster's website
Z3r0



Joined: 12 Jul 2005
Posts: 8

PostPosted: Wed Jul 13, 2005 8:56 am    Post subject: Reply with quote

lol sorry, it was late last night and I wanted to sleep,
Peter B. Clements over at quickpar (par2) suggested that a par3 was in progress that could process the data faster than par2 but file format would be the same, is this what your project is? par3? format the same as par2? Could you please clarify?

I want to be able to select say 10 files and be able to have enough recovery blocks to recover 2 files for instance, like in quickpar and therefore reduce inefficiency.

yes, it is quite hard to see, when you create / verify a data set which files are ok and which aren't as it appears right at the bottom of the screen in a tiny console, perhaps you could have it say what number of blocks were found and if any weren't next to the file name?

could we right click on files and directories to create ecc / verify ecc files inside and outside of the program (on the desktop)

The part of the interface I found hard to deal with when i first saw it was the create ecc files part as it's just a mass of boxes in no particular order, maybe you could use tabs or just rearrange it a bit so it's less scary to newbies Smile

Instead of printing out all the blocks which are damaged could it just state the number of blocks as it's pretty hard to scroll up and count lots of blocks

Also for me: Quickpar is faster at creating the recovery blocks, this is on a set of 21 WMA files 50mb each in size and 21 M4A files 5mb each in size, 1 JPG 50kb and 1 PNG 1mb total size approximately 673MB
Your program runs at 450Mb/s and quickpar runs at 650Mb/s, strange I know
2:24 for ECC to create @ 450Mb/S 1:40 for Quickpar to create @ 650Mb/s
Verify times are approximately the same at about 30 seconds.

I can't actually do a fast ECC test, clicking on the button doesn't do anything.
Back to top
View user's profile Send private message
ICE Graphics
Site Admin


Joined: 31 Mar 2003
Posts: 430

PostPosted: Wed Jul 13, 2005 1:27 pm    Post subject: Reply with quote

Z3r0 wrote:
lol sorry, it was late last night and I wanted to sleep,
Peter B. Clements over at quickpar (par2) suggested that a par3 was in progress that could process the data faster than par2 but file format would be the same, is this what your project is? par3?

No. ICE ECC is independent project. However it's use the same mathematical basis. The same class of Reed-Solomon codes using.

Z3r0 wrote:
format the same as par2? Could you please clarify?

Distinctions between formats PAR2 and ECC are huge. To list similar features easier, than to list all distinctions. Format PAR2 theoretically cannot be adapted to keep compatibility with the previous versions PAR2 and to have functionality ICE ECC format.

Z3r0 wrote:
I want to be able to select say 10 files and be able to have enough recovery blocks to recover 2 files for instance, like in quickpar and therefore reduce inefficiency.

As i understand you want to have partial recovery. To recover not all damaged files. It's possible, but anyway you must have enough recovery blocks for recover all damaged files, even if you want to recover only one damaged block. This is not ICE ECC limitation. It demands the theory of restoration of data.

Z3r0 wrote:
yes, it is quite hard to see, when you create / verify a data set which files are ok and which aren't as it appears right at the bottom of the screen in a tiny console, perhaps you could have it say what number of blocks were found and if any weren't next to the file name?

ICE ECC does not show exactly witch files are damaged. In difference from PAR2 at ICE ECC in one block can be any number of files. If you have a lot of files, to show list of all damaged files can be not good idea. Now ICE ECC recovers all damaged files. In what case to show list of exactly damaged files not necessarily.


Z3r0 wrote:
could we right click on files and directories to create ecc / verify ecc files inside and outside of the program (on the desktop)

Planning in future versions.


Z3r0 wrote:
The part of the interface I found hard to deal with when i first saw it was the create ecc files part as it's just a mass of boxes in no particular order, maybe you could use tabs or just rearrange it a bit so it's less scary to newbies Smile

OK

Z3r0 wrote:
Instead of printing out all the blocks which are damaged could it just state the number of blocks as it's pretty hard to scroll up and count lots of blocks

Not need to count. After finishing verifies ICE ECC show how much blocks are damaged if recovery is not possible. But if recovery is possible, ICE ECC just does recovery.

Z3r0 wrote:
Also for me: Quickpar is faster at creating the recovery blocks, this is on a set of 21 WMA files 50mb each in size and 21 M4A files 5mb each in size, 1 JPG 50kb and 1 PNG 1mb total size approximately 673MB

21x50 = 1050MB. Total size can not be 673MB.

Z3r0 wrote:
Your program runs at 450Mb/s and quickpar runs at 650Mb/s, strange I know
2:24 for ECC to create @ 450Mb/S 1:40 for Quickpar to create @ 650Mb/s
Verify times are approximately the same at about 30 seconds.

What the CPU are you using? How much RAM? What type of RAM? What parameters of recovery data are you using? What is block count and count of recovery blocks?

This is very strange result. Possible you made some mistake, like you made mistake with calculating total size. I made some test. Data almost the same size, like you told:

21 MP3 file - size 59311332 x 21
21 MP3 file - size 5227414 x 21
1 JPG file - size 49484
1 PNG file - size 947919
Total size of data: 1 356 311 069

506 blocks, 50 recover blocks

QuickPar create: 04:41
ICE ECC create: 03:47
ICE ECC create + verify: 04:43

Z3r0 wrote:
I can't actually do a fast ECC test, clicking on the button doesn't do anything.

It's do test, but it's very fast. You can see result almost immediately.
Back to top
View user's profile Send private message Visit poster's website
bobby89



Joined: 20 May 2005
Posts: 9

PostPosted: Wed Jul 13, 2005 6:52 pm    Post subject: Reply with quote

ICE Graphics wrote:


Z3r0 wrote:
I want to be able to select say 10 files and be able to have enough recovery blocks to recover 2 files for instance, like in quickpar and therefore reduce inefficiency.

As i understand you want to have partial recovery. To recover not all damaged files. It's possible, but anyway you must have enough recovery blocks for recover all damaged files, even if you want to recover only one damaged block. This is not ICE ECC limitation. It demands the theory of restoration of data.



Hi,


No I think he means that he want to know and be sure of it that by creating 30 blocks will be enough to recover 2 files. The only way to be sure is to calculate himself:

- 10 files you want to protect
- 2 files are the maximum you want to recover

Each file are 10Mb.

In that case this guy want to create at least 20Mb of ECC blocks to be able to recover 2 files. But if you don't pay attention or you make a bad math you are going to create only 19Mb os ECC blocks and then in the worst case where 2 files would be completely damaged, you could not recover.

QuickPar has an indication status of how much file can be recovered at minimum and maximum.

That option is only needed for usenet users, because on usenet we post a set of files identical in size. If you protect you personal data, each file is different in size, this option is then totally useless.
Back to top
View user's profile Send private message
ICE Graphics
Site Admin


Joined: 31 Mar 2003
Posts: 430

PostPosted: Wed Jul 13, 2005 7:22 pm    Post subject: Reply with quote

bobby89 wrote:
No I think he means that he want to know and be sure of it that by creating 30 blocks will be enough to recover 2 files. The only way to be sure is to calculate himself:

- 10 files you want to protect
- 2 files are the maximum you want to recover

Each file are 10Mb.

In that case this guy want to create at least 20Mb of ECC blocks to be able to recover 2 files. But if you don't pay attention or you make a bad math you are going to create only 19Mb os ECC blocks and then in the worst case where 2 files would be completely damaged, you could not recover.

QuickPar has an indication status of how much file can be recovered at minimum and maximum.

Is possible to add what indication. But i am not sure what it's really need to do. See below.


bobby89 wrote:
That option is only needed for usenet users, because on usenet we post a set of files identical in size. If you protect you personal data, each file is different in size, this option is then totally useless.

In PAR2 any file start with begin of block. But in ICE ECC using continious blocks. All data files concatenated like in TAR file, and data protection apply to this is TAR file. So, if you want to guarantee recovery some quantity of files and all files has the same size, you can do it easy, without any calculations. Set block size equal to file size. Of course all files must to have the same size. Then number recovered blocks mean number of maximum files witch is possible to recover.
Back to top
View user's profile Send private message Visit poster's website
Z3r0



Joined: 12 Jul 2005
Posts: 8

PostPosted: Thu Jul 14, 2005 6:51 pm    Post subject: Reply with quote

hi yes sorry, they're about 25MB in size the WMA files
Also number of blocks 1024 for both quickpar and ice eec and recovery blocks 102 for both
CPU athlon 64 3200+ socket 939 winchester
Ram 2x256MB DDR400 geil

I have 10 files on DVD, of varying size, the largest being say 5MB and 4MB, I create par2 files of 9MB size therefore being able to recover AT LEAST the 2 LARGEST files and X amount of small files. This is what I use quickpar for.
I do this so that if there are sector errors and the 2 largest files are not readable then I can recover them. (i.e. if 1 sector is non-recoverable by the DVDs built in error repairing routines the file may not be readable at all and a CRC error will occur and no write will occur to the hDD)
Also it probably would be better for me to make a sector by sector copy of the DVD, mount the ISO in daemon-tools and recover the damaged file to the HDD like that as more sectors would be readable but that's a last resort. Smile

Please can you describe the best way to protect my music? I usually try to keep perhaps 10/20% recover data or enough to recover 2 large files but as you state ICE-EEC works differently and this may not be possible? basically I don't want to have redundant data on my DVD as this is a waste of space.

p.s. is it possible to recover files to another folder in ICE-EEC I haven't checked?

fyi
4,321,840, 2,822,082, 4,156,101, 3,120,576, 3,432,299, 4,637,461, 3,671,064, 3,682,780, 3,407,487, 2,181,591, 3,897,378, 4,573,996, 5,095,570, 8,258,934, 3,334,314, 4,048,412, 4,049,337, 2,614,352, 3,443,729, 5,586,626, 2,558,780 mp4s

27,944,831, 19,163,637, 26,537,201, 21,804,667, 21,818,165, 29,339,099, 23,064,923, 24,995,427, 22,850,399, 15,342,783, 24,982,137, 30,290,969, 32,851,603, 53,725,741, 22,542,073, 27,636,425, 25,786,477, 16,562,773, 21,040,565, 35,117,317, 16,763,879 wmas

1,401 playlist
29,438 jpg
4,138,789 png

46 File(s) 627,225,428 bytes
Back to top
View user's profile Send private message
ICE Graphics
Site Admin


Joined: 31 Mar 2003
Posts: 430

PostPosted: Fri Jul 15, 2005 11:27 am    Post subject: Reply with quote

Z3r0 wrote:
hi yes sorry, they're about 25MB in size the WMA files
Also number of blocks 1024 for both quickpar and ice eec and recovery blocks 102 for both
CPU athlon 64 3200+ socket 939 winchester
Ram 2x256MB DDR400 geil

New tests for your data sizes:

QuickPar - Create: 03:45
ICE ECC - Create: 03:13
ICE ECC - Create + Fast Test: 03:25
CPU Athlon 2500. I do not think what AMD64 too different from AMD32.

Hint: when you test speed do not forget to clear system cache


Z3r0 wrote:
Please can you describe the best way to protect my music? I usually try to keep perhaps 10/20% recover data or enough to recover 2 large files but as you state ICE-EEC works differently and this may not be possible? basically I don't want to have redundant data on my DVD as this is a waste of space.

You are planning to protect info from damage on DVD disk. You not need to worry about number of files witch are possible to recover, because DVD disk is sector based device. The two main parameters most important for your case: number of blocks (or block size) and redundancy. The more the number of blocks, the more errors can be recovered. Because DVD errors usually not too large. Defective sectors will alternate with normal sectors. It is fair both for scratchs, and for degradation of a surface. My suggestion: select redundancy level as you want (10-25%). After what select block count maximum as possible for you. Growth of number of blocks increases time of calculations. Therefore you need to determine only two parameters: redundancy and time which you are ready to spend for making ECC files.


Z3r0 wrote:
p.s. is it possible to recover files to another folder in ICE-EEC I haven't checked?

Yes. When you start "Verify", ICE ECC will ask you about path for data files and recovery path. If this is path are different, ICE ECC will copy data to recovery directory. Data files in what mode do not modify.
Back to top
View user's profile Send private message Visit poster's website
Z3r0



Joined: 12 Jul 2005
Posts: 8

PostPosted: Fri Jul 15, 2005 7:17 pm    Post subject: Reply with quote

ok cool thanks for the info, will try some more tests in a few days time Smile
Back to top
View user's profile Send private message
Z3r0



Joined: 12 Jul 2005
Posts: 8

PostPosted: Thu Jul 28, 2005 5:55 pm    Post subject: Reply with quote

version 2.0
ICE-EEC creates in 1 minutes 20 seconds @ 750 MB/s

I don't know why but my system is working faster now and anyway
Quickpar creates in 1 minute 40 seconds @ 600 MB/s

very big improvement well done Smile
Back to top
View user's profile Send private message
mrQQ



Joined: 08 Jul 2005
Posts: 10

PostPosted: Tue Aug 02, 2005 5:12 am    Post subject: Reply with quote

Quote:
As i understand you want to have partial recovery. To recover not all damaged files. It's possible, but anyway you must have enough recovery blocks for recover all damaged files, even if you want to recover only one damaged block. This is not ICE ECC limitation. It demands the theory of restoration of data.


so you mean if I have 100mb of data, 10mb of ECC files, and 20mb of damaged data in that 100mb, I can't even recover the other 10mb? So that in the end I could have 90mb correct? that's a damn pity..
Back to top
View user's profile Send private message
ICE Graphics
Site Admin


Joined: 31 Mar 2003
Posts: 430

PostPosted: Tue Aug 02, 2005 6:25 am    Post subject: Reply with quote

mrQQ wrote:
so you mean if I have 100mb of data, 10mb of ECC files, and 20mb of damaged data in that 100mb, I can't even recover the other 10mb? So that in the end I could have 90mb correct? that's a damn pity..

If you have 100mb data, 10mb of ECC files, and 20mb of damaged data - YOU CAN NOT RECOVER EVEN 10mb. Recover is possible only if there are more recovery blocks than damaged blocks.

P.S. It's not ICE ECC limitation. It's pure mathematic.
Back to top
View user's profile Send private message Visit poster's website
mrQQ



Joined: 08 Jul 2005
Posts: 10

PostPosted: Tue Aug 02, 2005 11:56 am    Post subject: Reply with quote

damn i didnt realize that.. and there is nothing that can be done about that? :/
Back to top
View user's profile Send private message
ICE Graphics
Site Admin


Joined: 31 Mar 2003
Posts: 430

PostPosted: Tue Aug 02, 2005 5:12 pm    Post subject: Reply with quote

mrQQ wrote:
damn i didnt realize that.. and there is nothing that can be done about that? :/

Nothing
Back to top
View user's profile Send private message Visit poster's website
mrQQ



Joined: 08 Jul 2005
Posts: 10

PostPosted: Tue Aug 02, 2005 8:54 pm    Post subject: Reply with quote

i wasn't talking about "algorithmic" solution to the problem (i believe you that it isn't possible). i was thinking more like a workaround for it (as an option, of course). didnt have much time to go through ideas, but perhaps something like this could work..

say same example - 100mb data, 10% redundancy, 20mb damaged data.

what if we split source data in 10 sets of 10mb - S1..S10. We get 10 sets of redundant data aswell - R1..R10, each 1mb (those can be compiled into one file perhaps, or left separate, doesnt matter). Now, this doesnt help at all if damaged data is continous, as for one set, there is only 10% of redundancy. But if damage is scaterred , it could perhaps be possible to save some data? Now that I think about it, it doesnt sound so easy.. but perhaps a combination of split-set redunancy with full-set redundancy could work? This is just an idea for work-around, i'm sure you or someone else can come up with something even smarter!

p.s. thanks for implementing user suggestion so fast! it's rare to see such good customer support and rapid application development! all the suggestions i sent you to mail i found in next version! nice.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    www.ice-graphics.com Forum Index -> Suggestions All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group