To CCK El Khawarizmi
Good day
I am Wadï Mami, a computer engineer (ENSI class of 2000).
I'm sharing my idea for sequencing all data ( example DNA and so whatever) in the shortest possible time.
The idea is to use the Java Spring Batch framework to sequence a portion of the database table Data ,
then run multiple instances of this program in parallel to sequence the entire Database table ( see picture above).
This concept can be used in parallel computing across various fields, such as mathematics and
physics pushing parallel computing to the limits.
To break through the limitation of 20 parallel executions in Spring Batch 2.0, you must modify your
XML or Java configuration to override the framework's internal multi-threading defaults and resource
pools.
Now Spring Batch 6 eliminates previous parallel processing limitations by introducing a completely
rewritten chunk-oriented concurrency model based on a bounded producer-consumer pattern.
Here I will speak about one node think if we have a grid of Pcs or servers. I will use a matrix of SB
(spring Batch) allowed by RAM memory and disk space the same thing goes for jdk (RAM
availability and disk storage) I create the following
[PathToJDK1]/jdk1
[PathToJDK2]/jdk2
[PathToJDK3]/jdk3
[PathToJDK4]/jdk4
[PathToJDK5]/jdk5
.
.
.
.
.
.
[PathToJDKn]/jdkn
On every [PathToJDKi]/jdk"i"/bin folder copy spring Batch executable program file generated by mvn
-package command lets call it SB1 for the first one and then copy it into multiple SB program so we
have for every jdki/bin the following
[PathToJDKi]/jdk"i"/bin/SB1
[PathToJDKi]/jdk"i"/bin/SB2
[PathToJDKi]/jdk"i"/bin/SB3
[PathToJDKi]/jdk"i"/bin/SB4
[PathToJDKi]/jdk"i"/bin/SB5 .
.
.
.
[PathToJDKi]/jdk"i"/bin/SBn
here n is the number limit allowed by a single jvm jdk"i" memory limit to execute your multiple SB
java program instance.
Then launch all in parallel with different parameters for all Spring Batch instances on all JDK/bin
folders like the following in a batch file commmand
[PathToJDKi]/jdki/bin/java SBl -version"il" like
so we have
[PathToJDK1]/jdk1/bin/java SB1 -"11"
[PathToJDK1]/jdk1/bin/java SB2 -"12"
[PathToJDK1]/jdk1/bin/java SB3 -"13"
.
.
.
[PathToJDK1]/jdk1/bin/java SBn -"1n"
[PathToJDK2]/jdk2/bin/java SB1 -"21"
[PathToJDK2]/jdk2/bin/java SB1 -"22"
[PathToJDK2]/jdk2/bin/java SB3 -"23"
[PathToJDK2]/jdk2/bin/java SB4 -"24"
....
[PathToJDK2]/jdk2/bin/java SBn -"2n"
...
..
.
.
.
[PathToJDKi]/jdki/bin/java SBl -"il"
.
..
. [PathToJDKi]/jdki/bin/java SBn -"in"
..
etc
-- Minds, like parachutes, function best when open. ,,,
(o o)/ --------oOO--(_)--OOo--------------------\
| Wadï Mami didipostman
| Github : https://www.github.com/didipostman
| e-mail : wmami@steg.com.tn / didipostman77@gmail.com
| ----------------------------------------/
Comments
Post a Comment